Re: stereoscopy
Ichthyostega <
prg@...>
2010-01-01 23:11:31 GMT
Seb schrieb:
> let me first introduce myself. i am a 19 year old student from germany...
> i would like to ask whether there is any ambition to integrate stereoscopy
> into your program since this is the obvious future of creating movies.
Hi Sebastian,
be welcome to Lumiera!
As you probably know, I am one of Lumiera's core developers and working
on the "middle layer", where the editing operations done by the user
will work with the objects in the current session. I am a 40+ years old
software developer and living in munich, southern germany.
Speaking for myself, 3D isn't just a feature. Rather, stereoscopy and
spatial sound are about the primary motivations why I decided that we
need better editing tools. Not that our current tooling is broken,
but it contains seemingly innocuous pre-decisions, which effectively
drive you into handling some aspects of film-making in a ever repeating
standard-way. So it just looks as being most rational, when you
"stick to the standards"
Relating to the topic in question here, it works out as being plain
irrational if you want to build upon the full spatiality of sound for
your soundtrack; you will endanger the deadline of the project, or
at least risk ruining your health by excessively working overtime.
It is just so plain flat more rational to do a plain flat "panned mono"
soundmix, and then make it "pop" after the fact by adding proper effects.
Thus, what /would be possible/ going the other route remains undiscovered.
Panned mono (and flat imagery) gets all the resources, all the training
all the mastery of the seasoned experts of the field and the reviewers.
Here I've mentioned spatial sound first, because I guess much more
folks have hit this invisible wall when concerned with sound. But the
situation is completely similar, if not even worse when it comes to
spatiality in the imaging. Personally, coming from painting and
photography, at some point around 1991 I've almost completely switched
over to stereoscopy -- and I've learned the lesson that you can't just
add an "3D" effect to an conventional image composition. The whole
nature of composition is changed. And on top of this artistic challenge
comes the fact, that, the moment you switch to 3D, the existing tooling
starts working against you.
Now the fact that a fully professionally oriented editing solution is
lacking for the Linux platform gives us the chance to re-do some of
the basic decisions. We don't have to re-invent everything from scratch,
just re-visit the basic concepts, re-investigating if they support such
a new ascent, or rather work out as an impediment.
For sake of completeness I should mention that Lumiera has a bunch of goals.
None of them is strictly "codified" of course. The project is open and will be
moved by those dedicating their time and efforts.
Certainly one of the goals of Lumiera is to be prepared for the technological
advancements. What does it mean to work with 2k, 4k, 8k? How important is
hardware acceleration (really), and how can it be made fit in into the
existing environment? What benefits can we take from the advent of massively
multi-core hardware?
Besides that, another goal within the Lumiera project can be called "building
bridges". Classical film editing is an very deep, very minimal, almost ascetic
art. There is the real danger of this getting buried within an environment
which is totally focussed on effects. A mentality which thinks, steady-cams
and blue screens are what make a movie cool. To the contrary, Lumiera shall
be an environment, where the good old art of editing can be exercised,
without being hostile to the new achievements.
> ...I decided to get some grip onto this filming techniqe. I have been
> following the talk in the mailinglist and some updates on your website but
> still i don't know whether your concept allows implementing such a feature.
> if so i would like to offer trying to implement it. i have got some
> programming skills and it would be worth a try. but first let me know what
> you think about it!
Of course we'll welcome such an effort. We have an ambitious concept and
thus far we've just ignored the reality (i.e. the available manpower) and
worked on implementing this concept step by step. Of course we're cutting
edges here and there, because we do need some integration and a working
end-to-end chain. But up to now we didn't sacrifice any of the fundamental
goals and we didn't add a solution which would endanger the concept.
Now you're probably interested especially how this relates to stereoscopy.
Personally, I've always considered 3D imaging and sound, and I came to the
conclusion, that supporting it creates special requirements on two different
levels: openness/awareness in the core, and special extension modules, which
need to be first-class citizens.
Within the core, 3D just needs to be an option. This may seem trivial, but it
isn't. It means, that 3D can be chosen the same way, as you can choose to do
your project with RGB or with YUV+alpha-floats. All relevant parts simply
need to be prepared, that -- just by virtue of project configuration -- there
could be 2 channels or 5 channels or 21 channels or even a complicated mixture
of channels with different media types. Thus, branching and decisions have to
be modularised and factored out, and the backbone of the application has to
be written at an (sometimes quite demanding) level of abstraction. Especially,
this is demaning, as too much generality can kill pretty much every software
development project.
Following this general approach, in my design of the session's internal
structures I've settled down on some solutions, which I want to point out
in this context: To start with, I've removed most of the traditional meaning
of a "track".
You know, the classical association is a track on a magnetic tape or even
the film strip itself. It contains the data, or the data is piped through
the track. Then you can hook up your processing on top of it.
I've done away with this concept. In Lumiera, a track is just a space
(or a scope) where you can place media objects (along a time axis, that is).
It's the job of the engine to wire up these objects in a way which is in
accordance to the object's media type. The usual separation between "audio"
and "video" is gone. A clip is just a clip and contains a combination of
channels. If you want to treat sound separately, then make a clip just
comprised of sound and put it somewhere; the system will understand no
"video out" connection is necessary.
Another decision I want to mention here is to make some relations between
media objects into first class citizens: It is very common to help the editor
with arranging the elements by providing "snap-to" and alignment GUI functions.
E.g. the GUI is manipulated in a way that it makes a second clip snap in at the
end of the first clip. I think, it's a better approach not to implement such
fundamental things by GUI trickery. Rather, the information of a relation
between these two clips ("seamlessly following each other") is retained within
the data model of the session. Even if you'd be modifying the session through a
script, without any GUI, this relation will be retained. Indeed, I even went one
step farther and sort-of "turned around" the model: In Lumiera, the model is
built up from *relations* (called "Placement") and the actual media objects are
attached as leaves to these relations.
Together, these changed approaches give us the necessary flexibility to
handle 3D as "an option", without making the model more complicated.
Indeed, my goal is even overall to get a more simple model and session
representation, as a lot of special cases are replaced with one single
rule of treatment.
On top of this session core handling, the specific treatment of 3D image and
sound then has to happen within special plug-ins. But these plug-ins aren't
just add-ons, rather, Lumiera was designed from start to be modularised
into plug-ins starting at a certain level. For example, all of the actual
data processing will happen in plug-ins. Regarding 3D, the most notable
plug-ins are "panning/spatialisation" for sound and something I'd like
to call "windowing" for stereoscopy. Together, they serve to build up a
complete and smooth spatial image. They share the common property, that
they need to be inserted into the processing chain very much "downstream",
when entering the global busses, while the controlling parameters are very
much "upstream", at the individual media, or even more on the enclosing
scopes (tracks, sequences, global rules). In my concept, the Placements
serve as the glue to hold this control chain together. But honestly, this
latter part is just an concept and not designed in detail. I'm continuously
working on it and would welcome discussion.
Now I've written quite some details and I hope I didn't confuse you completely.
I'd be happy to explain more details, so feel free to ask questions or start
a discussion. Maybe it would be a good idea to "meet" on IRC ? We're using
channel #lumiera on freenode.net. (But currently I'm quite busy and not normally
hanging out at IRC, so please drop me a mail and then we could meet there to
discuss more details).
Cheers,
Hermann Voßeler
aka. "Ichthyo"