Git transition

Re: LyX Chat

Il 28/12/2011 23:23, Tommaso Cucinotta ha scritto:
> Il 28/12/2011 19:31, Pavel Sanda ha scritto:
>> how hard would it be to enhance your patch so collaborate editing of
>> lyx document is possible?
>
> I thought to this just a little: from my patch, we can merely reuse the
> socket set-up and non-blocking polling structure for avoiding multiple
> threads. However, the structure of a collaborative editor should be
> quite different, as instead of transmitting document fragments, it
> should actually transmit users' actions.
>
> Luckily, LyX has a quite enforced separation between the GUI and
> the model, i.e., the (in)famous LFUN dispatching mechanism. Therefore,
> what is needed is simply a "RemoteView", with a client and a server part.
> The client part, instead of (or in addition to) dispatching the LFUN to
> the local view, serializes the whole LFUN (let's neglect commands
> inserting external material such as graphics, for now) through the 
> socket,
> where the server part will periodically poll other user's LFUNs and
> dispatch actually them to the local view (local to the remote user  ).
> This way you would have the
> user acting on his/her own local document, and the remote user
> copying the same actions on his/her own remote document.
>
> I'm only thinking to 2 users, not 3 or more, but the logic should not be
> that different.

If you want to see just a preliminary hack that kind of works, please,
find it attached.

The BufferView has been extended so as to allow for incoming connections
from a remote LyX instance, as well as to connect to a remote LyX instance.

All LFUNs dispatched to the view are also copied to the remote instance, and
any LFUN coming from it is dispatched locally.

In order to try the patch, you need to:

1) launch LyX, and create a new buffer
2) issue the "collaborate-bind 8888" command in the mini-buffer (8888 is 
the TCP port, you can use whatever you prefer)
3) launch another LyX, and create a new buffer
4) issue the "collaborate-connect 127.0.0.1 8888" command in the 
mini-buffer (change destination host and port as needed)

Now the two views are kept synchronized, i.e., any LFUN dispatched to 
one of them is forwarded to the other.

I don't pretend the design to be anyway correct, just a hack made for 
fun .

While working on it, some weird things happens to fall under my eyes, 
such as why in GuiView there is this
dispatchToBufferView() method that tries to dispatch to the buffer view 
and after to its cursor, but why
doesn't the BufferView::dispatch() method itself dispatch to its own 
cursor ?

Btw, any comment is welcome.

     T.

ps: happy new year !!!!!

Re: LyX Chat

Il 02/01/2012 10:18, Tommaso Cucinotta ha scritto:
>
> If you want to see just a preliminary hack that kind of works, please,
> find it attached.

I forgot to mention: the files in qt4/ are still part of the Chat 
capability patch.

For the collaborative editing, all the affected files are in src/support 
and src/.
> I don't pretend the design to be anyway correct, just a hack made for 
> fun .

Specifically, I think we need absolutely 2 views over the same buffer, 
not just one,
in such a case , i.e., when accepting a connection from remote, we should
create a new BufferView. Similarly, when connecting to a remote LyX, we 
should
create a new BufferView. Additionally, who initiates a connection to a 
remote
instance, should also download from it the whole buffer over which to 
work in
a new WorkArea. All things quite easy to add...
Also, mouse-related LFUNs just do not make sense to be forwarded remotely,
as they refer to mouse coordinates which are specific to the local instance.
However, they should be someway inhibited, otherwise the 2 views go easily
out of sync. Such things are more difficult to achieve, as due to the number
of LFUNs existing in LyX.

     T.

PATCH: for 6760 and 7884

I've attached patches for two tickets:

http://www.lyx.org/trac/ticket/6760 and
http://www.lyx.org/trac/ticket/7884

If nobody objects I'd like to commit them.

Best wishes and a happy new year to all of you.

Stephan

Re: LyX Chat

On 02/01/2012 10:36, Tommaso Cucinotta wrote:
> Il 02/01/2012 10:18, Tommaso Cucinotta ha scritto:
>>
>> If you want to see just a preliminary hack that kind of works, please,
>> find it attached.
>
> I forgot to mention: the files in qt4/ are still part of the Chat 
> capability patch.
>
> For the collaborative editing, all the affected files are in 
> src/support and src/.
>> I don't pretend the design to be anyway correct, just a hack made for 
>> fun .
>
> Specifically, I think we need absolutely 2 views over the same buffer, 
> not just one,
> in such a case , i.e., when accepting a connection from remote, we 
> should
> create a new BufferView. Similarly, when connecting to a remote LyX, 
> we should
> create a new BufferView. Additionally, who initiates a connection to a 
> remote
> instance, should also download from it the whole buffer over which to 
> work in
> a new WorkArea. All things quite easy to add...
> Also, mouse-related LFUNs just do not make sense to be forwarded 
> remotely,
> as they refer to mouse coordinates which are specific to the local 
> instance.
> However, they should be someway inhibited, otherwise the 2 views go 
> easily
> out of sync. Such things are more difficult to achieve, as due to the 
> number
> of LFUNs existing in LyX.

No, not if you create a new BufferView. In this case all mouse related 
action needs to be forwarded as well. As a bonus, you will be able to 
see exactly what the remote colleague is doing. But this BufferView 
shall be read-only for the local guy. If you want to edit this Buffer, 
you have to create a new editable BufferView on the same Buffer (which 
will be forwarded as read-only to the remote guy). In this new WorkArea, 
you'll be able of course to see the remote Buffer change. You can now 
split the view to see the two BufferView at the same time.

In order for this scheme to work smoothly, you'll need first to let the 
remote BufferView be handled in a dedicated thread. Second, all methods 
that modify the Buffer contents needs to be protected by mutexes. All in 
all, a lot of small work but not very difficult.

That could be the flagship feature of LyX-2.1 

Abdel.

Re: LyX Chat

On 27/12/2011 23:06, Tommaso Cucinotta wrote:
> Hi all,
>
> I've just realized a small "chat" feature within LyX. If anyone is 
> interested, have a look at its description and patch here:
>
>   http://www.lyx.org/trac/ticket/7964
>
> Any comment is welcome.

This is interesting (but less than you collaboration patch).
First comment: please use QTcpSocket and signal/slot connections.

I have a pending patch that introduces Qt signals in Buffer and 
BufferView, I'll try to cleanup and commit that one.

> Bye,
>
>     T.
>
> ps: happy (holi)days to everyone!!

Happy new year to everyone!

Abdel.

Re: LyX Chat

Il 02/01/2012 14:54, Abdelrazak Younes ha scritto:
> This is interesting (but less than you collaboration patch).

nice to hear  Btw, in both cases, what is really missing, is a LyX 
collaboration server,
i.e., a simple server to be hosted somewhere (e.g., lyx.org ?) through 
which registered users
can collaborate (i.e., it merely receives messages and forwards them) 
even though their host
is not reachable from the Internet, and their firewalls do not allow any 
incoming connection
(but it should allow outgoing connections on the "LyX port" [to be 
identified], in order to allow
this to work of course).

> First comment: please use QTcpSocket and signal/slot connections.

I'll have a look, thanks. Another immediate question is whether there's 
any portable serialization framework already at reach for LyX (i.e., in 
terms of dependencies -- e.g., I could identify 
boost::serialize::archive & related). That may be useful to serialize 
FuncRequest objects, as well as other types of messages that may be 
worth to be transmitted through the socket(s).

> I have a pending patch that introduces Qt signals in Buffer and 
> BufferView, I'll try to cleanup and commit that one.

Ok, I don't know what changes will be introduced by this patch of yours, 
anyway, please, drop a message on the list about the implied changes on 
the code.

     T.

Re: LyX Chat

Il 02/01/2012 14:47, Abdelrazak Younes ha scritto:
> No, not if you create a new BufferView. In this case all mouse related 
> action needs to be forwarded as well.

what I mean, is that locally I can have no clue (and I should not have 
it) about the window and view geometry of the remote LyX instance. 
Therefore, if the remote user clicks or selects with the mouse, then 
these coordinates cannot make any sense for me. On the other hand, if 
the remote LyX instance decodes the mouse action by using its own 
metrics etc., then it forwards to me an equivalent (non-mouse) LFUN 
(e.g., an LFUN commanding to issue a Cursor::putSelectionAt() encoding 
someway the desired cursor position and selection length), then I'm 
perfectly able to execute it locally.

> As a bonus, you will be able to see exactly what the remote colleague 
> is doing. But this BufferView shall be read-only for the local guy. If 
> you want to edit this Buffer, you have to create a new editable 
> BufferView on the same Buffer (which will be forwarded as read-only to 
> the remote guy). In this new WorkArea, you'll be able of course to see 
> the remote Buffer change. You can now split the view to see the two 
> BufferView at the same time.

I'm not sure I understood what you suggested here.
>
> In order for this scheme to work smoothly, you'll need first to let 
> the remote BufferView be handled in a dedicated thread.

That's surely useful for avoiding that a network temporary delay causes 
my LyX instance to hand while receiving an LFUN from remote.

> Second, all methods that modify the Buffer contents needs to be 
> protected by mutexes. All in all, a lot of small work but not very 
> difficult.

This is not strictly needed, as my current scheme (periodic polling) may 
still be reused, by having the thread receiving from the socket push the 
received LFUN on a local LFUN queue, protected by a mutex, then LyX 
could simply periodically poll (in a non-blocking way) from such a 
queue, then issue the LFUNs locally. This allows to not spread mutexes 
across the whole LyX code, i.e., keep its current single-threaded 
orientation and still be able to receive and send through the remote 
socket(s). Only the LFUN queue would need mutex protection .

Btw, I have nearly another patch ready which supports multiple connected 
peers, and creates a different BufferView for each new remote peer 
collaborating.

Bye,

     T.

Re: LyX Chat

On 02/01/2012 16:53, Tommaso Cucinotta wrote:
> Il 02/01/2012 14:54, Abdelrazak Younes ha scritto:
>> This is interesting (but less than you collaboration patch).
>
> nice to hear  Btw, in both cases, what is really missing, is a LyX 
> collaboration server,
> i.e., a simple server to be hosted somewhere (e.g., lyx.org ?) through 
> which registered users
> can collaborate (i.e., it merely receives messages and forwards them) 
> even though their host
> is not reachable from the Internet, and their firewalls do not allow 
> any incoming connection
> (but it should allow outgoing connections on the "LyX port" [to be 
> identified], in order to allow
> this to work of course).

Unless we buy some server I don't think this will be possible. Maybe 
someone wants to deliver this kind of services for free or not... it 
does not have to be a LyX.org service I guess.

>
>> First comment: please use QTcpSocket and signal/slot connections.
>
> I'll have a look, thanks. Another immediate question is whether 
> there's any portable serialization framework already at reach for LyX 
> (i.e., in terms of dependencies -- e.g., I could identify 
> boost::serialize::archive & related). That may be useful to serialize 
> FuncRequest objects, as well as other types of messages that may be 
> worth to be transmitted through the socket(s).

IIRC, quite some time ago, Lars wanted to use this boost serialiser, try 
to look in the archives for "serialize any", or something.

>
>> I have a pending patch that introduces Qt signals in Buffer and 
>> BufferView, I'll try to cleanup and commit that one.
>
> Ok, I don't know what changes will be introduced by this patch of 
> yours, anyway, please, drop a message on the list about the implied 
> changes on the code.

It's just a patch that converts Buffer::changed() and all to Qt signals 
and so gets rid of WorkAreaManager and GuiDelegates.

Abdel.

>
>     T.
>

Re: LyX Chat

Il 02/01/2012 17:00, Abdelrazak Younes ha scritto:
>
>>
>>> I have a pending patch that introduces Qt signals in Buffer and 
>>> BufferView, I'll try to cleanup and commit that one.
>>
>> Ok, I don't know what changes will be introduced by this patch of 
>> yours, anyway, please, drop a message on the list about the implied 
>> changes on the code.
>
> It's just a patch that converts Buffer::changed() and all to Qt 
> signals and so gets rid of WorkAreaManager and GuiDelegates.

Ok, so, unless my memory is being unfaithful to me:
1) we dropped boost::signal() for compilation performance reasons 
(AFAICR, someone said that it was pulling 7K lines of code into each 
compilation unit)
2) we introduced WorkAreaManager and other equivalent features via 
custom dedicated classes
3) now we go back to signals, but in the Qt flavour.

This time I guess we don't have the problems of boost::signal, because 
Qt is much more polymorphic oriented, rather than being template-oriented ?

However, I guess the immediate drawback will be that each class willing 
to issue signals, will have to be a derivate of QObject, and have in its 
declaration some weird Qt macros, such as Q_OBJECT and similar ?
AFAICR, the original boost signals didn't have any constraints/impact 
like that on the code.

Just to know, thanks.

     T.