Stanley M. Ho | 11 Oct 2008 00:33
Picon

JSR 277 spec lead changes

Hi JSR 277 experts,

This is to let you know that today is my last day at Sun. It has been a 
great pleasure to work with all of you, and I really appreciate the 
contributions all of you have made in this JSR.

Alex Buckley will take over the responsibility for JSR 277, and serve as 
the sole spec lead. I have been working with Alex to make sure the 
transition is smooth. If you have any questions or concerns about the 
JSR, please contact alex.buckley@... directly.

Going forwards, I'm moving to Research In Motion. I'm reachable on 
Linkedin. Hopefully, many of you will stay in touch and our paths will 
cross again.

- Stanley

Bryan Atsatt | 13 Aug 2008 01:22
Picon
Favicon

Module change detection...

There are a number of class loading optimizations (including HotSpot's Class Data Sharing) that are hard to make generic due to the lack of a standard way to validate cached class state. We have a unique opportunity to fix this in 277...

I propose that we add:

1. An annotation to record a digest over the module's contents, which would then be visible from ModuleInfo.
2. A convenience api to return the digest, e.g.:
public interface ModuleContent {
    ...
    public byte[] getDigest();
}

The jam tool would compute and add the annotation, and the default implementation(s) of ModuleContent would then just return the value from the annotation.

Other implementations would be free to compute the value at install time (or even runtime); development environments could "generate" a value using a simple counter.

Unlike the version number, this value can be used to ensure that any change to the module's contents can be detected. Both HotSpot and other JVMs can then introduce optimizations like Class Data Sharing that depend on being able to validate the cache file.


It may similarly be useful to have ModuleInfo be able to (lazily) compute a digest for itself, and/or have a digest on ModuleDefinition that encompasses both the metadata (ModuleInfo) and the contents.

// Bryan





Bryan Atsatt | 8 Jul 2008 19:14
Picon
Favicon

Deployment module export...

I believe we need a mechanism to extract a deployment module from a 
Repository. This would enable:

1. Provisioning systems to copy from a remote repository to a local 
instance.
2. Migration of all content from one repository to another.
3. Backup of repository content regardless of implementation.
4. Tooling to gather dependent deployment modules into a larger 
deployment unit.

Given that Repository.install() currently takes a URL parameter, it 
would be nicely symmetrical if Repository could produce a URL (for a 
given ModuleDefinition) that could be passed directly to install(). For 
Repositories that act as a front end for a remote store, returning such 
a URL may be a little extra work, but the underlying mechanisms are 
already required. For local Repositories, if we recommend that 
implementations keep the original deployment module as a separate file, 
then a simple "file:" url can be returned. (Modulo the usual unspecified 
valid lifetime issue inherent in most URLs.)

Thoughts?

// Bryan

Bryan Atsatt | 11 Jun 2008 22:11
Picon
Favicon

ImportDependency

The design of this class seems broken to me with the addition of two 
"types". Why not:

1. Make ImportDependency abstract, removing the getType() and newX methods.
2. Make ImportPackage and ImportModule subclasses, each with their own 
newX method.

(I also strongly suggest that we rename ImportDependency to simply 
"Import" :^)

This model also obviously enables new subtypes, an extensibility point 
that may come in handy in the future.

// Bryan

Bryan Atsatt | 11 Jun 2008 03:27
Picon
Favicon

Refactorability proposal...

(FYI: I've discussed this with Stanley but wanted to ensure it was 
visible to everyone.)

Proposal

1. Eliminate runtime use of import-by-module in the JAM system:

    a. Support the (very convenient) import-by-module at the source level.
    b. jam tool transforms import-by-module to a list of 
import-by-package statements.

2. Add APIs to fully support import-by-package in the JAM system:

   a. Support Version annotation in package-info.java (or in 
module-info.java). If a
       package does not declare a version, it "inherits" that of the 
enclosing module.
   b. Add ImportPackage annotation with version constraints.

3. Add APIs to fully support import-by-package in the abstract framework:

    a. Add methods to Query to produce import-by-package nodes.
    b. Replace Query.getIndexableNames() with fully generic variants (I 
proposed
        a solution here previously which I will re-post).

Rationale

Module refactoring is inevitable, particularly during the transition 
from the current, effectively flat class space to a fine-grained space 
provided by module systems. We have significant experience with this 
issue at Oracle (with the transition to our own module system), and OSGi 
best-practices for conversion include starting with everything in one 
bundle and then separating out pieces as experience is gained.

A very common pattern, in our experience, is for developers to start 
with many extra jars in their initial module (a mini version of 
class-path hell). As that module is put into wider use, someone 
discovers that package X is also contained in their module, and that 
duplication either leads to runtime conflicts (very bad), or just plain 
footprint bloat. The obvious answer is to put package X in a separate 
module, and have everyone share it via imports.

But... not so fast. If there are consumers of that module who import it 
by module name alone, then pulling X out of it will cause those 
importers to break. And if it is possible for your module to have been 
imported by *anyone* by name alone, then you are stuck: either you break 
them or you live with the incorrect granularity (which just isn't an 
option in the conflict scenarios). Not a happy choice.

Originally, I had proposed to do away with import-by-module altogether, 
both to avoid this problem and to eliminate the conceptual disconnect. 
Your code does not today contain import statements that name *jars*, it 
names packages and/or specific classes in those packages. Why invent a 
new system that takes such a large step backwards?

The answer is simply convenience. Imagine a module that contains 100 
packages and it is obvious that writing a single import statement is far 
easier than discovering and explicitly writing all the package imports. 
Yes, IDEs will likely mostly eliminate this issue, but it still makes 
sense to be able to do this by hand.

This proposal is an attempt to maintain the convenience while adding the 
crucial ability to safely refactor: step 1b is the central idea.

// Bryan

Adrian Brock | 10 Jun 2008 13:57
Picon
Favicon

Exported Classes and OSGi Re: Supporting OSGi Bundles in the Java Module System

On Mon, 2008-04-28 at 20:16 -0700, Stanley M. Ho wrote:
> 3.2 ModuleDefinition class
> Two new methods are added in the ModuleDefinition class to return the
> exported packages and member packages respectively. The export and
> member definitions contained in the OSGi metadata are in package
> granularity. In addition, a new PackageDefinition class is added to
> allow an OSGi bundle to expose the metadata for an exported package.
> This is required to meet the requirements (1) and (2). 
>   ModuleDefinition class:
>       public abstract Set<PackageDefinition> getExportedPackageDefinitions();
>       public abstract Set<PackageDefinition> getMemberPackageDefinitions();
>       
>   PackageDefinition class:
>       public abstract class PackageDefinition {
>           public String getName();
>           public Version getVersion();
>           public Set<String> getAttributeNames();
>           public String getAttribute(String name); 
>       }

ModuleDefinition still refers to exported and member classes.
Although this is probably more of an issue for 294/291 interoperation,
I don't think 291 has such notions?
--

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Adrian Brock | 10 Jun 2008 13:52
Picon
Favicon

Re: Supporting OSGi Bundles in the Java Module System

On Mon, 2008-04-28 at 20:16 -0700, Stanley M. Ho wrote:
> 3.1.2 Constraint Checking
> A new method ModuleSystem.getModules is added to allow a ModuleSystem
> implementation to instantiate Module instances for multiple
> ModuleDefinitions in the same resolution and also to enforce
> constraints specified in these Modules. 
>   ModuleSystem class:
>       /**
>        * Returns the list of Module instances for the imports 
>        * in the same resolution for the specified importer.
>        * The returned Module instances are instantiated and initialized
>        * using the algorithm specific to this ModuleSystem.
>        *
>        * This method is called by a ModuleSystem when initializing
>        * a Module instance (importer) that imports Modules (imports) 
>        * from this ModuleSystem.
>        */
>       public List<Module> getModules(ModuleDefinition importer,
>                                              List<ModuleDefinition> imports)
>           throws ModuleInitializationException;
>   
> OSGi allows to put constraints on the importer through the metadata
> for an exported package in another bundle. The OSGi module system can
> enforce the constraints on the importer in the implementation of this
> getModules method. The importer can be a ModuleDefinition from other
> ModuleSystem. 
> 
> For example, the Apache Xerces XML parser bundle in Example 1 of
> Section 1 exports the org.apache.xerces.parsers package whose export
> definition has a "use" constraint. This export definition puts a
> constraint on the importer of the org.apache.xerces.parsers package to
> use the org.apache.commons.logging package wired to the
> org.apache.xerces.parsers package in the same resolution. The 1.0
> version of the Wombat application imports the
> org.apache.xerces.parsers bundle and the org.apache.derby bundle. The
> org.apache.xerces.parsers package will get wired to the version 1.0.4
> logging package and the constraint is satisfied and thus it can be
> wired successfully. 
> 
> Let's say a new version of the Wombat application (say version 1.2) is
> updated and it depends on an additional Apache Commons Logging utility
> which is also an OSGi bundle. 
> 
>   //
>   // com/wombat/app/module-info.java
>   //
>    <at> Version("1.2") 
>    <at> ImportModules({
>        <at> ImportModule(name="org.apache.xerces.parsers", version="2.6.6+")
>        <at> ImportModule(name="org.apache.derby", version="10.0+")
>        <at> ImportModule(name="org.apache.commons.logging", version="2.0+")
>   })
>   module com.wombat.app;
>   
> The bundle manifest header for the Apache Commons Logging utility is: 
>   org.apache.commons.logging:
>      Bundle-SymbolicName: org.apache.commons.logging
>      Bundle-Version: 2.0
>      Export-Package: org.apache.commons.logging; version=2.0
>   
> The 1.2 version of the Wombat application imports the
> org.apache.commons.logging bundle that violates the constraint if the
> org.apache.xerces.parsers package is wired to the version 1.0.4
> logging package but the Wombat application requires the version 2.0
> logging package. The version 1.2 of the Wombat application should fail
> to initialize due to this constraint. In other words, the getModules()
> method of the OSGi module system should throw a
> ModuleInitializationException when the OSGi module system determines
> that the constraint is violated. 

I assume the list of "importers" is intended to take the place
of Glynn's ResolutionContext?

On Mon, 2007-02-26 at 09:58 +0000, Glyn Normington wrote:
> Yes, if we want to support first class interoperation. Without
> something
> like a resolution context, I don't see how we can maximise sharing of
> dependencies between module systems. If we don't do that, we'll get
> all
> the usual symptoms of inconsistent class spaces, like class cast
> exceptions, broken loading constraints, etc. 

Assuming that is its purpose, I think the api is incomplete?

>From Glynn's original examples what is needed is to not to just look at
direct imports from the same module system.
Instead, to maximise sharing and consistency it needs to know about 
all other resolutions in a given top level resolution request.

I made a similar point on Glynn's original prototype:

On Mon, 2007-02-26 at 12:26 +0100, Adrian wrote: 
> On Mon, 2007-02-26 at 09:58 +0000, Glyn Normington wrote:
> > So the default behaviour in AbstractWireFactory.create calls
> > createLocal first so that the local name space takes precedence over
> > those of other module systems. The rationale is that the current
> > module system is like a local scope and we don't want other module
> > systems coming along and subverting the value associated with a given
> > module name that is defined locally.
> >
> 
> But equally, we want two module systems using the same module
> where possible.
> 
> e.g. If we are configured to have both a 277/291 repository
> and both define log4j:1.2.8 then the default mechanism
> should choose one of them, NOT each module system
> prefers its own.

I made another point (but I can't find the original e-mail :-) 
where I thought it probably wasn't enough to just pass
the modules in the current resolution process (i.e.
those just being resolved).

Instead, when resolving a module, there should be some
"ClassLoading Space" where the resolution can determine
all other modules that are already related due to
imports/exports (including transient ones).
--

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Adrian Brock | 10 Jun 2008 13:28
Picon
Favicon

Goals was: Re: Supporting OSGi Bundles in the Java Module System

On Mon, 2008-04-28 at 20:16 -0700, Stanley M. Ho wrote:
> 2. Requirements
>      1. It shall be possible for an OSGi container to implement the
>         Java Module System.
>      2. It shall be possible for a JAM module to express an import
>         dependency on any Module Definition in any Java Module System.

These goals appear to have changed from the ones we agreed?
I am misreading it or is this document only expressing one side of 
the equation?

On Fri, 2007-02-16 at 13:08 -0800, Stanley M. Ho wrote: 
> Goals:
> 
> 1. It shall be possible for JSR-277 modules to make use of modules from
> other module systems (e.g. JSR-291) (i.e. if other module systems expose
> their modules as JSR-277 modules through a repository implementation.)
> 
> 2. It shall be possible for other module systems (e.g. JSR-291) to be
> implemented on top of the JSR-277 APIs such that the modules from other
> module system can make use of JSR-277 modules.
> 

I originally read (2) as a rewording of Glynn's original
definition where the goal is to create a more peer orientated
delegation.

> On Fri, 2007-01-12 at 09:49 +0000, Glyn Normington wrote:
> > Michal Cierniak <cierniak@...> wrote on 09/01/2007 20:20:12:
> > Co[u]ld you write a one-paragraph outline of what "first class
> > interoperation with JSR 291" would mean?
> 
> First class interoperation with JSR 291 means that JSR 291
> implementations
> should be implementable on top of Java 7 APIs such that JSR 291
> bundles can
> make effective use of JSR 277 modules and vice versa.
> 
> (I guess JSR 291 modules residing in JSR 277 repositories would be
> helpful
> too, especially if anyone needs to create a hybrid application
> containing
> modules of both sorts.)

My critisms of using the JSR277 repositories to do the integration
included:

* There are already lots of orthogonal reasons for swapping
repository implementations

On Mon, 2007-02-26 at 12:26 +0100, Adrian wrote:
> So besides the peer/hierarchy argument there
> is also the problem that the repository is dealing
> with too many concerns.
> 
> 1) ModuleDefinition construction
> 2) Model - parent/child or peer
> 3) QoS - what tools are available for the repository
> 4) Location - file system/url based, etc.
> 5) Delegation - links to standard repositories
> etc. 

* Developing a full repository just to define a different archive
format is too much (e.g. legacy javaee deployments)

On Wed, 2007-02-28 at 17:12 +0100, Adrian wrote:
On Mon, 2007-02-26 at 18:05 -0800, Stanley M. Ho wrote:
> > If I understand you correctly, your primary concern is around
> requiring
> > other module systems which support 277 modules that are stored
> > differently (e.g. war, ear, etc.) to implement the classloading and
> > other functionalities from scratch. Is this accurate?
> 
> Not just that, anybody that wants to integrate their own
> packaging format and construct module definition (wrappers)
> should be able to plugin without dictating
> or developing a full repository.
> 
> The examples so far discussed are:
> OSGi modules
> JavaEE deployment packages 

* The concern about hierarchical integration

On Wed, 2007-02-28 at 17:18 +0100, Adrian wrote: 
> On Mon, 2007-02-26 at 18:05 -0800, Stanley M. Ho wrote:
> > Hi Adrian,
> >
> > > The hierarchical assumption of repositories makes it difficult
> > > to plugin peer modules, unless we are going to define
> > > a composite repository for this purpose.
> >
> > Can you give me an example of peer modules that you are concerned with?
> >
> 
> It is really peer module systems.
> 
> e.g. My repository is made up of modules from
> 
> 1) a local JSR77 repository
> 2) an OSGi repository handled by my OSGi "module system"
> 3) Some JavaEE deployments handled by the appserver's "module system"
> 4) a link to a repository on the internet
> 
> I want my JavaEE modules to be able to use OSGi modules
> and vice versa OSGi modules should be able to use my JavaEE modules
> (e.g. an appclient or a resource adapter)
> 
> This requires a peer integration at the classloader and security
> level.
--

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Michal Cierniak | 10 Jun 2008 08:39
Picon
Favicon

Re: Supporting OSGi Bundles in the Java Module System

Hi Stanley,

Thank you for providing this proposal.  I think that it looks quite
reasonable.  Once we have an implementation (are you working on one?),
we should have a look at details like: what exactly the behavior for
circular dependencies should be (I don't think there is a problem --
it's just one of those things where it's possible to overlook details
until you start to implement something).  But I think that the overall
proposal is a pretty good approach to this very important problem of
interop.

I'm curious what the OSGi experts think.

Michal

On Mon, Apr 28, 2008 at 8:16 PM, Stanley M. Ho <Stanley.Ho@...> wrote:
> Dear 277 experts,
>
> The first attachment is a draft spec for supporting OSGi bundles in the Java
> Module System. This is based on the past EG discussion in particular the
> proposals and inputs from Glyn, Richard, and Bryan. This is a work in
> progress and is expected to evolve based on further inputs from this EG.
>

Gordon Hirsch | 29 Apr 2008 16:43
Picon
Favicon

Re: Supporting OSGi Bundles in the Java Module System

Stanley,

The javadoc zip file was stripped by our firewall. Could you either 
resend with a different extension or post it to the EG homepage at jcp.org?

Stanley M. Ho wrote:
> Dear 277 experts,
> 
> The first attachment is a draft spec for supporting OSGi bundles in the
> Java Module System. This is based on the past EG discussion in
> particular the proposals and inputs from Glyn, Richard, and Bryan. This
> is a work in progress and is expected to evolve based on further inputs
> from this EG.
> 
> The second attachment is the latest API javadoc for your reference.
> 
> We're currently updating the JSR 277 EDR2 specification and APIs to make
> the distinction between the framework/abstractions for the Java Module
> System more clear, and we expect they will be available for the EG to
> review in a few weeks after JavaOne. In the meantime, the draft is
> intended to provide enough information for the EG to review and provide
> inputs without the EDR2.
> 
> Numerous EG members are preparing to attend and/or speak at JavaOne next
> week, so some of them may well not respond until afterwards.
> 
> Finally, thanks to Bryan and his blog that serves really well to set up
> the context for this discussion.
> 
> - Stanley and Alex
> 
> 
> ------------------------------------------------------------------------
> 
> 
>   Supporting OSGi Bundles in the Java Module System
> 
> *
> 
> Mandy Chung and Stanley Ho
> Version 0.1
> April 28, 2008
> Copyright 2008 Sun Microsystems, Inc.
> 
> * JSR 277 Early Draft defines the framework for the Java Module System 
> and provides the API to access Java modules such as searching for 
> modules, loading and initializing a module, and also managing its life 
> time. JSR 277 enables one or more module systems run on a single Java 
> virtual machine. This document^1 defines how JSR 277 supports OSGi 
> bundles as defined in JSR 291. This document is a work in progress and 
> is expected to evolve based on more input from the Expert Group.
> 
> Section 1 describes various definitions in the Java Module System. 
> Section 2 describes the requirements for supporting OSGi bundles in the 
> Java Module System and section 3 describes the proposed API changes. 
> Section 4 specifies how to map an OSGi bundle to a ModuleDefinition so 
> they can be consumed by other modules in the Java Module System. Section 
> 5 describes the repository mechanism that enables the OSGi module system 
> to be plugged into the Java Module System framework to make OSGi bundles 
> accessible by other module systems at runtime. Appendix A shows the code 
> example illustrating how a JAM module system implementation uses the JSR 
> 277 API to search and import OSGi bundles.
> 
> ^1 This draft is built upon from the basis of the past EG discussions, 
> proposals and suggestions for the interoperability support especially 
> the ideas and invaluable inputs from Glyn Normington, Richard Hall and 
> Bryan Atsatt.
> 
> 
>     0. Current Status
> 
> The JSR 277 EDR2 is being updated to make the distinction between the 
> framework/abstractions for the Java Module System and the JAM module 
> system (See Section 1) clearer.
> 
> This section highlights the main items that remain to be sorted out.
> 
>    1. Versioning scheme (see Section 4.2)
>    2. Security
>    3. Java Module Events
> 
> 
>     1. Definitions
> 
> /Java module/
> 
> A development abstraction that exists at compile-time in the Java 
> programming language and is reified in the Java Virtual Machine.
> 
> /Module Definition/
> 
> A deployment abstraction that contains metadata, classes and resources, 
> and is reified as ModuleDefinition as part of a Java Module System 
> implementation.
> 
> /JAM module/
> 
> A concrete, portable deployment format which implements the Module 
> Definition. Amongst other details, it has metadata, classes, resources, 
> and can contain embedded JAR files and native libraries. The 
> distribution format is called JAM (JAva Module) that is based on the JAR 
> file format.
> 
> /Java Module System/
> 
> A set of runtime abstractions that includes ModuleSystem, Repository, 
> ModuleDefinition (see "Module Definition" above), Module, 
> ImportDependency and others.
> 
> /JAM Module System/
> 
> A concrete implementation of the Java Module System that supports JAM 
> modules. It is the default implementation of the Java Module System for 
> the Java SE platform.
> 
> /OSGi Module System/
> 
> A concrete implementation of the Java Module System by an OSGi container.
> 
> 
>     2. Requirements
> 
>    1. It shall be possible for an OSGi container to implement the Java
>       Module System.
>    2. It shall be possible for a JAM module to express an import
>       dependency on any Module Definition in any Java Module System.
> 
> Below provides an example to illustrate how a JAM module imports OSGi 
> bundles.
> 
> A Wombat application is a JAM module that processes shopping orders for 
> a website. It depends on the Apache Xerces XML parser and the Apache 
> Derby and both of them are available as OSGi bundles. The module 
> definition of the Wombat application looks like this:
> 
>   //
>   // com/wombat/app/module-info.java
>   //
>    <at> Version("1.0") 
>    <at> ImportModules({
>        <at> ImportModule(name="org.apache.xerces.parsers", version="2.6.6+")
>        <at> ImportModule(name="org.apache.derby", version="10.0+")
>   })
>   module com.wombat.app;
>   
> 
> This Wombat application is packaged as a JAM file named 
> "com.wombat.app-1.0.jam" and the OSGi bundles it depends on are packaged 
> as JAR files containing the following manifests,^2 per the OSGi 
> specifications:
> 
>   org.apache.xerces.parsers:
>      Bundle-SymbolicName: org.apache.xerces.parsers
>      Bundle-Version: 2.9.10
> 
>      Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging",
>                      org.apache.xerces.jaxp; version=2.6.6,
>                      org.apache.xerces.framework; version=2.6.6,
>                      org.apache.xerces.readers; version=2.6.6,
>                      org.apache.xerces.utils; version=2.6.6,
>                      org.apache.commons.logging; version=1.0.4
>      Import-Package: javax.xml.parsers; version=1.2.0,
>                      org.w3c.dom; version=1.0.0,
>                      org.xml.sax; version=2.0.1
>      Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional
>                      
>   org.apache.derby:
>      Bundle-SymbolicName: org.apache.derby
>      Bundle-Version: 10.0.2
>      Export-Package: org.apache.derby.authentication,
>                      org.apache.derby.database,
>                      org.apache.derby.io,
>                      org.apache.derby.jdbc
>      Import-Package: java.sql
>   
> 
> ^2 These example manifests are for illustration purpose and they do not 
> represent the manifest in the actual Apache Xerces XML Parser and Apache 
> Derby bundles.
> 
> 
>     3. Proposed API Changes
> 
> This section describes the proposed API changes for supporting OSGi 
> bundles in the Java Module System.
> 
> 
>       3.1 ModuleSystem class
> 
> The ModuleSystem specification is updated to support multiple 
> implementations running in a single Java virtual machine. This update is 
> necessary to meet the requirement (1). See JSR 277 EDR2 for the full 
> specification.
> 
> A ModuleSystem implementation is responsible for creation, management, 
> and release of Java Modules owned by this module system. The 
> ModuleSystem specification does not define the resolution algorithm and 
> the distribution format that a module system supports. Instead, the 
> resolution algorithm and the distribution format are specific to a 
> ModuleSystem implementation.
> 
> 
>       3.1.1 Module Initialization
> 
> This section is a clarification to the JSR 277 EDR about the 
> initialization of a Module instance.
> 
> The ModuleSystem.getModule(ModuleDefinition) method returns a /fully 
> initialized/ Module instance^3 for a given ModuleDefinition. A Module 
> instance is fully initialized if the following conditions are all met:
> 
>    1. Its import dependencies have been resolved. ModuleDefinitions for
>       the imported Java modules satisfying the import dependencies and
>       all its constraints are found using the algorithm defined by this
>       ModuleSystem.
>    2. Module instances for its imports are successfully instantiated and
>       initialized by its owning ModuleSystem.
>    3. This ModuleSystem has performed its own type consistency checking
>       successfully in the Module instance.
>    4. If this ModuleSystem supports an initializer^4 to be invoked
>       before a Module instance is fully initialized, the initializer for
>       this Module instance is invoked.
> 
> In addition, a ModuleSystem can support import constraints specific to 
> its algorithm to tailor the resolution of the imports for a 
> ModuleDefinition. The constraints specified in a ModuleDefinition is 
> only known to its owning ModuleSystem and other ModuleSystem 
> implementations are not required to understand them.
> 
> ^3 Initializing an OSGi bundle is equivalent to resolving, wiring and 
> starting an OSGi bundle.
> 
> ^4 The ModuleSystem specification does not define a generic initializer 
> mechanism for ModuleDefinitions. The JAM module system supports the 
> module initializer through the ModuleInitializer API and the 
> ModuleInitializerClass annotation (see JSR 277 EDR 2). The OSGi module 
> system supports the module initializer through the bundle activator.
> 
> 
>       3.1.2 Constraint Checking
> 
> A new method ModuleSystem.getModules is added to allow a ModuleSystem 
> implementation to instantiate Module instances for multiple 
> ModuleDefinitions in the same resolution and also to enforce constraints 
> specified in these Modules.
> 
>   ModuleSystem class:
>       /**
>        * Returns the list of Module instances for the imports 
>        * in the same resolution for the specified importer.
>        * The returned Module instances are instantiated and initialized
>        * using the algorithm specific to this ModuleSystem.
>        *
>        * This method is called by a ModuleSystem when initializing
>        * a Module instance (importer) that imports Modules (imports) 
>        * from this ModuleSystem.
>        */
>       public List<Module> getModules(ModuleDefinition importer,
>                                              List<ModuleDefinition> imports)
>           throws ModuleInitializationException;
>   
> 
> OSGi allows to put constraints on the importer through the metadata for 
> an exported package in another bundle. The OSGi module system can 
> enforce the constraints on the importer in the implementation of this 
> getModules method. The importer can be a ModuleDefinition from other 
> ModuleSystem.
> 
> For example, the Apache Xerces XML parser bundle in Example 1 of Section 
> 1 exports the org.apache.xerces.parsers package whose export definition 
> has a "use" constraint. This export definition puts a constraint on the 
> importer of the org.apache.xerces.parsers package to use the 
> org.apache.commons.logging package wired to the 
> org.apache.xerces.parsers package in the same resolution. The 1.0 
> version of the Wombat application imports the org.apache.xerces.parsers 
> bundle and the org.apache.derby bundle. The org.apache.xerces.parsers 
> package will get wired to the version 1.0.4 logging package and the 
> constraint is satisfied and thus it can be wired successfully.
> 
> Let's say a new version of the Wombat application (say version 1.2) is 
> updated and it depends on an additional Apache Commons Logging utility 
> which is also an OSGi bundle.
> 
>   //
>   // com/wombat/app/module-info.java
>   //
>    <at> Version("1.2") 
>    <at> ImportModules({
>        <at> ImportModule(name="org.apache.xerces.parsers", version="2.6.6+")
>        <at> ImportModule(name="org.apache.derby", version="10.0+")
>        <at> ImportModule(name="org.apache.commons.logging", version="2.0+")
>   })
>   module com.wombat.app;
>   
> 
> The bundle manifest header for the Apache Commons Logging utility is:
> 
>   org.apache.commons.logging:
>      Bundle-SymbolicName: org.apache.commons.logging
>      Bundle-Version: 2.0
>      Export-Package: org.apache.commons.logging; version=2.0
>   
> 
> The 1.2 version of the Wombat application imports the 
> org.apache.commons.logging bundle that violates the constraint if the 
> org.apache.xerces.parsers package is wired to the version 1.0.4 logging 
> package but the Wombat application requires the version 2.0 logging 
> package. The version 1.2 of the Wombat application should fail to 
> initialize due to this constraint. In other words, the getModules() 
> method of the OSGi module system should throw a 
> ModuleInitializationException when the OSGi module system determines 
> that the constraint is violated.
> 
> 
>       3.2 ModuleDefinition class
> 
> Two new methods are added in the ModuleDefinition class to return the 
> exported packages and member packages respectively. The export and 
> member definitions contained in the OSGi metadata are in package 
> granularity. In addition, a new PackageDefinition class is added to 
> allow an OSGi bundle to expose the metadata for an exported package. 
> This is required to meet the requirements (1) and (2).
> 
>   ModuleDefinition class:
>       public abstract Set<PackageDefinition> getExportedPackageDefinitions();
>       public abstract Set<PackageDefinition> getMemberPackageDefinitions();
>       
>   PackageDefinition class:
>       public abstract class PackageDefinition {
>           public String getName();
>           public Version getVersion();
>           public Set<String> getAttributeNames();
>           public String getAttribute(String name); 
>       }
>   
> 
> The version and attributes in the PackageDefinition are optional and for 
> information only and to aid diagnosis. A ModuleSystem importing a 
> ModuleDefinition from other ModuleSystem is not required to understand 
> the version and attributes of its exported PackageDefinitions.
> 
> 
>       3.3 ImportDependency class
> 
> The ImportDependency class is updated as follows.
> 
>    1. An import type is added to indicate if the ImportDependency is for
>       module-level or package-level. "module" and "package" are two
>       defined import types.
>    2. The ImportDependency class can have attributes for a module system
>       to include additional information about an import dependency.
> 
> This is required to meet the requirement (1).
> 
>   ImportDependency class:
>          // static factory methods
>        public static ImportDependency 
>            newImportModuleDependency(String moduleName,
>                                      VersionConstraint constraint,
>                                      boolean reexport,
>                                      boolean optional,
>                                      Map<String, String> attributes);
>        public static ImportDependency
>            newImportPackageDependency(String packageName,
>                                       VersionConstraint constraint,
>                                       boolean optional,
>                                       Map<String, String> attributes);
>        public String getType();
>        public String getAttribute(String name);
>        public Set<String> getAttributeNames();
>   
> 
> 
>     4. Mapping OSGi Bundles to ModuleDefinitions
> 
> This section specifies how an OSGi bundle maps to a ModuleDefinition to 
> expose in the Java Module System so that other ModuleDefinitions can 
> import them.
> 
> 
>       4.1 Bundle-SymbolicName
> 
> The bundle symbolic name maps to a module name (i.e. 
> ModuleDefinition.getName()). The directives for the Bundle-SymbolicName 
> header maps to the module attributes.
> 
> For example:
> 
>       Bundle-SymbolicName: com.acme.foo
>   
> 
> The Java Module System and OSGi do not enforce any naming convention. It 
> is encouraged to use the reverse domain name convention to name OSGi 
> bundles and Java modules to avoid the name space conflict.
> 
> 
>       4.2 Bundle-Version
> 
> A bundle version maps to a module version^5 (i.e. 
> ModuleDefinition.getVersion()). Bundle-Version is an optional header and 
> the default value is 0.0.0. The bundle version format is:
> 
>       major[.minor[.micro]][.qualifier]
>   
> 
> The module version format is:
> 
>       major[.minor[.micro[.update]]][-qualifier]
>   
> 
> If the bundle version contains a qualifier, the delimiter prior to the 
> qualifier will need to be changed from a period ('.') to a dash ('-'). 
> For example, the bundle version 3.1.4.pi maps to the module version 
> 3.1.4-pi.
> 
> ^5 *Difference in OSGi and JSR 277 versioning scheme*
> The versioning scheme defined in the JSR 277 Early Draft is loosely 
> based on the existing versioning schemes that are widely used in the 
> Java platform today and for backward compatibility reason (see JSR 277 
> EDR chapter 5). Many existing products including the JDK use the version 
> format with the micro, minor, micro, and update numbers. A version with 
> no qualifier is higher than the version with the same version number but 
> with a qualifier. This is more intuitive and has been the convention the 
> JDK has been using. The Expert Group has discussed the difference with 
> the OSGi versioning scheme and agreed with the JSR 277 versioning scheme 
> defined.
> 
> *Open Issue:*
> Need to investigate the version mapping due to the difference in the 
> comparison of two versions - one with a qualifier and the other without 
> a qualifier.
> 
> When two bundle versions have the same major, minor, and micro numbers, 
> the bundle version that has a qualifier is lower than the bundle version 
> that has no qualifier. e.g. 7.8.9.pi < 7.8.9
> 
> When two module versions have the same major, minor, micro, and update 
> numbers, the module version that has a qualifier is higher than the 
> module version that has no qualifier. e.g. 7.8.9 < 7.8.9-b04-alpha
> 
> One possible solution would be:
> 
>   OSGi version                      JSR 277 Version
>   
>   major.minor.micro            ->   major.minor.micro-0
>   major.minor.micro.qualifier  ->   major.minor.micro-1-qualifier  
>   
> 
> 
>       4.3 Import-Package
> 
> The Import-Package header maps to the import dependencies for a Java 
> module (i.e. ModuleDefinition.getImportDependencies()).
> 
> The Import-Package header contains one or more import definitions, each 
> of which describes a single package import for a bundle. Each import 
> definition maps to an ImportDependency instance with the "package" type 
> as follows:
> 
>     * The import package name maps to ImportDependency.getName().
>     * ImportDependency.getType() returns "package".
>     * The "resolution" directive maps to the "optional" input parameter
>       of the ImportDependency constructor; true if
>       "resolution:=optional" is specified and false otherwise.
>     * The "version" attribute maps to the VersionConstraint of the
>       ImportDependency as described in the version-range section below.
>     * All other attributes specified in the import definition including
>       the bundle-symbolic-name and bundle-version attributes map to the
>       attributes in the ImportDependency.
> 
> Example,
> 
>       Import-Package: p;
>           version="[1.23, 1.24]";
>           resolution:=optional
>   
> 
> maps to the import dependencies equivalent to:
> 
>       ImportDependency importP = 
>           ImportDependency.newImportPackageDependency("p", 
>                                                       VersionConstraint.valueOf("[1.23,1.24]",
>                                                       true /* optional */);
>   
> 
> 
>         The version-range mapping:
> 
> The OSGi version-range maps to a VersionConstraint as follows:
> 
>     * If a version has no qualifier, the mapping is exact. For example,
>       a bundle version range [1.1, 1.2) maps to a module version range
>       [1.1, 1.2). If there is a qualifier, then section 4.2 should be used.
>     * If a bundle version range is specified as a single version, it
>       will map to an open version range. For example, the bundle version
>       range "1.23" maps to the module version range "1.23+".
> 
> 
>       4.4 Export-Package
> 
> The Export-Package header maps to the exported package definitions for a 
> Java module (i.e. ModuleDefinition.getExportedPackageDefinitions()).
> 
> The Export-Package header contains one or more export definitions, each 
> of which describes a single package export for a bundle. Each export 
> definition maps to a PackageDefinition instance as follows:
> 
>     * The package name maps to PackageDefinition.getName().
>     * The "include" and "exclude" directive along with the classes in
>       the exported package are the input to determine the returned value
>       of the ModuleDefinition.isClassExported() method.
>     * The "version" attribute maps to PackageDefinition.getVersion().
>     * Other attributes and directives including the "use" directive in
>       the export definition can map to the attributes in the
>       PackageDefinition.
>     * The exported package definition is also a member package
>       definition for the module.
> 
> 
>       4.5 Require-Bundle
> 
> The required bundles maps to the import dependencies for a Java module 
> (i.e. ModuleDefinition.getImportDependencies()). Each required bundle 
> maps to an ImportDependency instance with the "module" type:
> 
>     * The bundle symbolic name of the required bundle maps to
>       ImportDependency.getName().
>     * ImportDependency.getType() returns "module".
>     * The "visibility" directive maps to
>       ImportDependency.isReexported(). The isReexported() method returns
>       true if "visibility:=reexport" is specified; false otherwise.
>     * The "resolution" directive maps to ImportDependency.isOptional().
>       The isOptional() method returns true if "resolution:=optional" is
>       specified; false otherwise.
>     * The "bundle-version" attribute maps to the VersionConstraint of
>       the ImportDependency as described in the Version Range Mapping
>       section of section 4.4.
> 
> Example,
> 
>       Require-Bundle: com.acme.facade;visibility:=reexport,
>           com.acme.bar;visibility:=reexport;resolution:=optional
>   
> 
> maps to the import dependencies equivalent to:
> 
>       ImportDependency facade = 
>           ImportDependency.newImportModuleDependency("com.acme.facade", 
>                                                      VersionConstraint.DEFAULT,
>                                                      true /* reexport */,
>                                                      false /* optional */);
>                                                     
>       ImportDependency bar = 
>           ImportDependency.newImportModuleDependency("com.acme.bar", 
>                                                      VersionConstraint.DEFAULT,
>                                                      true /* reexport */,
>                                                      true /* optional */);
>   
> 
> 
>       4.6 Other Manifest Headers
> 
> The above sections cover the manifest headers that provide the metadata 
> for the OSGi resolver (see Section 3.5 of the OSGi Service Platform Core 
> Specification Release 4, Version 4.1). The other bundle manifest 
> headers, including Bundle-Vendor, Bundle-Description and 
> DynamicImport-Package, do not affect the module resolution. This 
> specification does not need to define how to map them to 
> ModuleDefinition. However, implementations are encouraged to include 
> them as the module attributes (i.e. ModuleDefinition.getAttribute()) as 
> additional information to aid diagnosis.
> 
> 
>       4.7 Fragment Bundles
> 
> Fragment bundles are not exposed as ModuleDefinitions in the Java Module 
> System. Instead, they are exposed as part of the ModuleDefinition of its 
> host bundle to which they are attached to.
> 
> 
>       4.8 Example
> 
> The manifest in the org.apache.xerces.parsers bundle shown in Example 1 
> of Section 2 is:
> 
>   org.apache.xerces.parsers:
>      Bundle-SymbolicName: org.apache.xerces.parsers
>      Bundle-Version: 2.9.10
> 
>      Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging",
>                      org.apache.xerces.jaxp; version=2.6.6,
>                      org.apache.xerces.framework; version=2.6.6,
>                      org.apache.xerces.readers; version=2.6.6,
>                      org.apache.xerces.utils; version=2.6.6,
>                      org.apache.commons.logging; version=1.0.4
>      Import-Package: javax.xml.parsers; version=1.2.0,
>                      org.w3c.dom; version=1.0.0,
>                      org.xml.sax; version=2.0.1
>      Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional
>   
> 
> Below shows the ModuleDefinition for this OSGi bundle when exposed in 
> the Java Module System. For clarity, we only show one Export-Package 
> entry and one Import-Package entry.
> 
> Method of ModuleDefinition 	Returned Value 	Bundle Manifest Header
> getName() 	"org.apache.xerces.parsers" 	Bundle-SymbolicName
> getVersion() 	Version.valueOf("2.9.10") 	Bundle-Version
> getImportDependencies() 	
> 
> ImportDependency.newImportModuleDependency("org.osgi.util.xml", 
>                                            VersionConstraint.valueOf("1.0.0+"),
>                                            true /* reexport */,
>                                            true /* optional */);
> 
> 	Require-Bundle
> 
> ImportDependency.newImportPackageDependency("javax.xml.parsers", 
>                                             VersionConstraint.valueOf("1.2.0+"),
>                                             false /* optional */);          
> 
> 	Import-Package
> getExportedPackageDefinition() 	PackageDefinition with:
> 
>  name="org.apache.xerces.parsers",
>  version="2.6.6"
>  attributes=(("uses", "org.apache.commons.logging")}
> 
> 	Export-Package
> 
> 
>     5. Enabling the OSGi Module System in the Framework
> 
> To enable the OSGi module system in the framework, an OSGi Repository 
> implementation should be plugged into the runtime. The OSGi repository 
> is responsible for discovering OSGi bundles and exposing them as 
> ModuleDefinitions so that OSGi bundles are available for other module 
> systems to use. A ModuleSystem implementation finds OSGi bundles via the 
> repository delegation model and therefore the OSGi repository has to be 
> configured as an ancestor of the repository where the Java module 
> depending on OSGi bundles resides. Otherwise, it will fail to find the 
> importing OSGi bundles.
> 
> One or more repositories can be created for the OSGi module system and 
> interoperate with other module systems via the repository delegation model.
> 
> The following picture depicts the repository tree set up to run the 
> Wombat application described in Example 1 of Section 2. The OSGi 
> repository is configured as the parent of the application repository. 
> This particular OSGi repository implementation includes the Apache Felix 
> OSGi runtime for loading and resolving OSGi bundles.
> 
>  
>                ------------------------
>                | Bootstrap Repository |
>                ------------------------
>                           |
>                           |
>                  --------------------
>                  | Global Repository |
>                  --------------------
>                           |   
>                           |                       xxxxxxxxxxxxxxxx
>                  -------------------              x Apache Felix x
>                  | OSGi Repository |   <====>     x     OSGi     x ---> org.apache.xerces.parsers version 2.10
>                  -------------------              x   Runtime    x      org.apache.derby version 10.0.2
>                           |                       xxxxxxxxxxxxxxxx      org.apache.derby version 9.1
>                           |
>                   ----------------- 
>                   |  Application  | 
>                   |  Repository   |
>                   -----------------
>                   com.wombat.app-1.0.jam
>   
> 
> For example, the following command will launch the Wombat application in 
> the "/wombat-application" directory.
> 
>                 
>   > java -repository /wombat-application -module com.wombat.app         
>   
> 
> The com.wombat.app module is located in the application repository which 
> is a repository for the JAM module system. The JAM module system first 
> loads the com.wombat.app module. To initialize this JAM module, the JAM 
> module system looks at its import dependencies and performs a search of 
> two imported OSGi bundles through the repository delegation model from 
> the OSGi repository. The JAM module system then requests the OSGi module 
> system associated with the OSGi repository to get the Module instances 
> for the imports (see Section 3.1). Once the imported OSGi bundles are 
> loaded and started, the initialization process for the com.wombat.app 
> module continues.
> 
> 
>       6. Delegation of Class and Resource Loading
> 
> The class loader of a Module instance (i.e. returned by the 
> Module.getClassLoader() method) must be capable to load all classes and 
> resources in the module.
> 
> As described in Section 5 above, when the com.wombat.app JAM module is 
> initialized, two other Module instances for its imports representing the 
> resolved OSGi bundles are created in the system. The class loader for 
> the Module instance for the org.apache.xerces.parsers bundle must be 
> capable to load all classes and resources in it. Similarly for the 
> org.apache.derby bundle.
> 
> 
>     7. Security
> 
> TBD.
> 
> 
>     8. Implementation Notes
> 
> The following are the notes for the implementation of the OSGi 
> repository and the implementation of the Java Module System.
> 
>    1. The repository delegation hierarchy is a tree and thus cycles
>       involving multiple module systems are inherently unsupported.
>    2. The repository delegation model is designed to offer isolation
>       between ModuleDefinitions in different repositories. Although a
>       module system could have access to multiple repositories, the
>       module system should adhere to the repository delegation module.
>       Otherwise, it would break the isolation model the repository
>       provides.
>    3. Java SE 7 is expected to have parallel class loading support. All
>       module systems plugged in the framework are required have parallel
>       class loading enabled in order to avoid potential deadlocks.
>    4. Split packages without shadowing are explicitly permitted in OSGi
>       whereas the JAM module system does not allow split packages. So
>       importing OSGi bundles with split packages in a JAM module will
>       result in module initialization failure.
>    5. When a resolution involves multiple module systems, a module
>       system implementation should take the possible potential issues
>       (such as hanging) into account in their design to prevent a
>       foreign module system from bringing down the module system or the
>       entire JVM. A module system could implement time out policy to
>       prevent from hanging the module system.
> 
> 
>     8. References
> 
>     * OSGi Service Platform Core Specification Release 4, Version 4.1
>       April 2007
>     * JSR 277 Interoperation with OSGi by Richard Hall and Glyn
>       Normington, Apr 24, 2006.
>     * Module System Interoperability by Richard Hall, May 11, 2006.
> 
> 
>     A. Appendix
> 
> The following illustrates how the JAM module system implementation uses 
> the JSR 277 API to search and import OSGi bundles This example does not 
> cover the exact resolution algorithm and the implementation of the 
> module system runtime.
> Example 1 of Section 1 has a com.wombat.app JAM module importing two 
> OSGi bundles.
> 
> com.wombat.app ----> org.apache.xerces.parsers
>                |
>                |---> org.apache.derby
> 
> // the Java runtime will first find the module com.wombat.app
> ModuleDefinition wombatModDef = repository.find("com.wombat.app", "1.0+");
> // This call blocks until com.wombat.app is fully initialized
> Module wombat = wombatModDef.getModuleInstance();
> 
> 	
> /JAM Module System Runtime/
> 
> // find imports for wombat
> List<ModuleDefinition> imports = ...;
> Map<ModuleSystem, List<ModuleDefinition>> foreignImportMap = new HashMap....;
> for (ModuleDefinition md : imports) {
>     if (md is from a foreign module system (ms)) {
>         // the case to add a new entry in foreignImportMap 
>         // is not shown in this pseudo-code.
>         // 
>         // org.apache.xerces.parsers and org.apache.derby will be added in this map       
>         foreignImportMap.get(ms).add(md); 
>     } else {
>         // JAM resolution algorithm
>         ...
>     }
> }
> 
> // Gets imports from foreign module systems
> // Should do this in a separate thread since this is a synchronous call
> for (ModuleSystem ms : foreignImportMap.keySet()) {
>    // The getModules() method will allow a module system to know if 
>    // a set of ModuleDefinition are resolved in the same resolution. 
>    List<Module> imports = ms.getModules(wombatModDef, foreignImportMap.get(ms));
>    // interconnect the imports with wombat
>    ...
> }
> 
> // continue the module initialization process such as shadow validatation
> // and execute initializers
> ...
> 
> // Module initialization completed
> if (succeeded) {
>    com.wombat.app is now fully initialized
> } else {
>    // signal the getModule method to throw ModuleInitializationException
>    ...
> }
> 

Stanley M. Ho | 1 May 2008 22:48
Picon

JavaOne

Hi JSR 277 experts,

Here is the schedule for JSR 277 and related talks by EG members at 
JavaOne next week:

* TS-6185: Modularity in Java™ Platform
Wednesday
5/7/2008
10:50 AM -11:50 AM

* BOF-5032: Modularity in the Java™ Platform
Wednesday
5/7/2008
8:30 PM -9:20 PM

* BOF-6180: Class Loader Rearchitected
Wednesday
5/7/2008
7:30 PM -8:20 PM

* TS-5122 Converting (Large) Applications to OSGi
BJ Hargrave, IBM; Peter Kriens, aQute
Thursday 5/8/2008
13:30 - 14:30

We know some of you will be at the conference, and we hope to meet you 
at the modularity session and BOF!

- Stanley and Alex


Gmane