SMIL 2.0 Extension for Professional Multimedia Authoring - Preliminary Investigation
Outline
- Background and Motivation: Why SMIL ?
- to enlarge potential SMIL market
- to innovate content production workflow
- Requirements for Professional Multimedia Authoring
- Material Identification
- Content Description
- Profiles, Conformance
- Improvement of SMIL 2.0
- Extension
- Addtion
- Re-definition, Clarification
- Re-consideration
SMIL: Current Status
- designed for multimedia content description on the Web
- 1998.6: SMIL 1.0 W3C Recommendation
- 2001.8: SMIL 2.0 W3C Recommendation
- technically in good shape
- precious resource on the Web
- on the market ..
- shortage (almost lack) of commercial SMIL contents
- multi-codec (mpeg-1/2/4,qt,real,win,..) capability is required,
rather than multimedia integration capability
-
standard DRM still not ready
- DVD already on the market
- Sony also seems one of suspects:|
- for now ..
- still under development on the commercial market
SMIL: Key to Success in Business
- success in the CE market is indispensable
- potential SMIL markets:
- Low-end: Mobile
- Midrange: PC, Web
- SMIL 2.0 ? XHTML+SMIL Profile ?
- High-end: TV, DVD, Digital Cinema, ..
- why not ? making the best use of SMIL for professional content business
- challenge: let SMIL go beyond the Web
- SMIL Professional Profile ???
- contents flow:
- Production-site -> Consumer-site
- in terms of professional content business:
- Consumer Needs < SMIL Capability < Production Needs
Innovation: Media Authoring Process
- Destructive Authoring - Conventional Approach
- Non-Destructive Authoring - Prospective Approach
- authoring descriptions usability, distributed authoring capability
Requirements for Professional Media Authoring Description Language
- Material Identification
- Unique Material Identifier (UMID)
- Media Track Identification
- Media Time Identification
- Temporal Duration Media Clipping
- Spatial Region Media Clipping
- Intermediate Media Identification
- Content Description
- Metadata Description
- Metadata Synchronization
- Media Transition
- Media Composition and Transformation
- Animated Subtitles
- Profiles, Conformance
- variety of products must work properly together
- in general: fidelity rather than interactivity or adaptivity
SMIL 2.0
- 10 Groups of Modules
- Structure Module
- Media Objects Modules
- Timing and Synchronization Modules
- Layout Modules
- Content Control Modules
- Linking Modules
- Animation Modules
- Metainformation Module
- Transition Effects Modules
- Time Manipulations Module
Professional Media Clipping
Requirement:
Temporal Duration Media Clipping
- media clipping capability by referring embedded SMPTE Time Codes (TCs)
Limitation:
Proposal:
- If the media have embedded time codes, the offset should be measured with the embedded time codes.
See next slide
Affect:
Media Objects Modules (Re-definition/Clarification)
Professional Media Clipping
CASE1:
<ref src="media1" clipBegin="smpte=01:02:00:00" clipEnd="smpte=01:06:00:00"/>
media1 |---------+++++++++++++++++++++---------| 8-minute-long
01:00:.. 01:02:00:00 01:06:.. 01:08:00:00 embedded TCs in media1
ref |+++++++++++++++++++| 4-minute-long
00:00:00:00 00:04:00:00
CASE2:
<ref src="media2" clipBegin="smpte=00:02:00:00" clipEnd="smpte=00:06:00:00"/>
media2 empty // |-------------------| 8-minute-long
00:02:.. 00:06:.. // 01:00:00:00 01:08:00:00 embedded TCs in media2
ref |---------| // 4-minute-long
00:00:.. 00:04:.. //
Media Type Casting
Requirement:
Media Track Identification
- extracting and handling individual types of media, e.g. video, audio, out of complex media
Limitation:
Proposal:
- video element should represent video, audio element should represent audio,
exclusively.
<par>
<video src="media1"/>
<audio src="media2"/>
</par>
Affect:
Media Objects Modules (Re-definition/Clarification)
Media Track Identification and Destination
Requirement:
Media Track Identification
- manipulating individual tracks within a media material
- a media material may consist of multiple tracks
Proposal:
by XPointer-like URI fragment:
<audio src="media1#track('A1')" cast="left"/>
by attribute:
<audio src="media1" track="A1" cast="left"/>
Affect:
Media Objects Modules (Extension)
Visual Area Clipping
Requirement:
Spatial Region Media Clipping
- media clipping capability by specifying coodinates
Proposal:
- apply coords attribute to media object elements and time container elements
<video src="media2" coords="0%,0%,50%,100%"/>
<seq coords="50%,0%,100%,100%">
<video src="media1"/>
<video src="media2"/>
</seq>
Affect:
Media Objects Modules, Timing and Synchronization Modules (Extension)
Media Transparency Modification
Requirement:
Visual Media Transparency (Alpha-Blending)
Proposal:
- apply alpha attribute to media object elements and time container elements
<par alpha="0.5">
<video src="media1" alpha="0.8"/>
<video src="media2" alpha="0.2"/>
</par>
Affect:
Media Objects Modules, Timing and Synchronization Modules (Extension)
Media Time Addressing
Requirement:
Media Time Identification
- embedding Time Codes (TCs) into resulting media
Proposal:
- apply SMPTE TCs to begin|end attributes
See next slide
Affect:
Timing and Synchronization Modules (Extension)
Media Time Addressing
<ref src="media1" begin="smpte=01:00:00:00" end="smpte=01:05:00:00"/>
media1 |+++++++++++++++++++|
03:00:00:00 03:04:00:00 embedded TCs in media1
01:00:00:00 01:04:00:00 applied TCs to ref element
<seq begin="smpte=01:00:00:00">
<ref src="media1"/>
<ref src="media2"/>
</seq>
media1 |+++++++++++++++++++|
03:00:00:00 03:04:00:00 embedded TCs in media1
media2 |+++++++++++++++++++|
02:00:00:00 02:04:00:00 embedded TCs in media2
seq |+++++++++++++++++++++++++++++++++++++++|
01:00:00:00 01:04:00:00 01:08:00:00 applied TCs to seq element
Time Container Clipping
Requirement:
Intermediate Media Identification
- improvement of Time Container (par|seq) usability as Media Object
Proposal:
- apply clipBegin|clipEnd attributes to time container elements
See next slide
Affect:
Timing and Synchronization Modules (Extension)
Time Container Clipping
<ref src="media0" clipBegin="smpte=01:02:00:00" clipEnd="smpte=01:06:00:00"/>
Given media0 as a par element below:
<par>
<ref src="media1" begin="smpte=01:00:00:00"/>
<ref src="media2" begin="smpte=01:03:00:00"/>
</par>
to be equivalent to a par element below:
<par clipBegin="smpte=01:02:00:00" clipEnd="smpte=01:06:00:00">
<ref src="media1" begin="smpte=01:00:00:00"/>
<ref src="media2" begin="smpte=01:03:00:00"/>
</par>
Metadata Description
Requirement:
Metadata Description
- inclusion of non-RDF XML descriptions
- e.g. MPEG-7 Descriptions
- may be introduced with rdf:parseType="Literal",
but this makes descriptions unnecessarily redundant
Proposal:
<head>
<metadata type="text/xml">
<mp7:mpeg7 xmlns:mp7="urn:mpeg:mpeg7:...">
....
</mp7:mpeg7>
</metadata>
<metadata type="application/rdf+xml">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
....
</rdf:RDF>
</metadata>
</head>
Affect:
Metainformation Module (Extension)
Metadata Synchronization
Requirement:
Metadata Synchronization
- metadata synchronization with audio/video streams
- e.g. MPEG-7 Binary Stream
Proposal:
- using ref element is possible but ambiguous
- should define another Media Object element ?
- audio|video|img|animation|text|textstream
- why not for metadata
-
meta|metadata elements are already in use for other purpose
-
metastream ?
Affect:
Media Objects Modules (Extension)
Media Transition
Requirement:
Transition Effects on Time Container Elements
- improvement of Time Container (par|seq) usability as Media Object
Proposal:
- allow transIn|transOut attributes to be applied to Time Container elements
Why not ?
<par transOut="crossfade">
<ref src="media1"/>
<ref src="media2"/>
</par>
rather than
<par>
<ref src="media1" transOut="crossfade"/>
<ref src="media2" transOut="crossfade"/>
</par>
Affect:
Transition Effects Modules (Extension)
Audio Transition
Requirement:
Audio Transition
- only visual behavior is defined, what about audio and other data ?
- should be clearly defined
Proposal:
- increase/decrease soundLevel along with transition progress
- needs more consideration
Affect:
Transition Effects Modules (Re-definition/Clarification)
Transformation Effects Modules
Requirement:
Media Composition and Transformation
Proposal:
- follow the syntax of Transition Effects Modules
BasicTransformations Module
declare transformation element in head element,
refer with transform attribute of media object element
InlineTransformations Module
apply transformationFilter element directly to Media Object Element
- transformation effect types
-
visualDeform -- mosaic, frosting and crystal effects
-
colorEffect -- monotone, sepia and negative
-
chromaKey -- blue-screen
-
soundEffect -- hall, theater and stadium
Affect:
SMIL 2.0 Modules (Addition)
Timed Text
Requirement:
Animated Subtitles
- single standard
- Title, Subtitles, Captions, Baloons
Proposal:
- W3C Timed Text Working Group will work on this !
Profiles, Conformance
Requirement:
lightweight profile and conformance
Proposal:
- slim-down of minimum profile
- SMIL 2.0 Basic Profile requires 11(10) modules
- Structure Module
- Media Object Modules -- BasicMedia
- Layout Modules -- BasicLayout
- Timing and Synchronization Modules -- BasicInlineTiming, MinMaxTiming, BasicTimeContainers, RepeatTiming,
(SyncbaseTiming)
- Content Control Modules -- BasicContentControl, SkipContentControl
- Linking Modules -- BasicLinking
- all is not necessary and too many from the professional content authoring viewpoint
- Tiny Profile ? Core Profile ?
- needs more consideration
Summary
- Background and Motivation
- to enlarge potential SMIL market
- to innovate content production workflow
- Requirements
- Media Identification
- Content Description
- Profiles, Conformance
- Improving SMIL 2.0
- Extensions, Clarifications and Additions
- lightweight profile and conformance to be investigated
- Comments and suggestions are welcome
- Let's resume W3C SMIL(SYMM) Working Group !
Thank you !