2/09/2008

Extending Maven SCM: proxy-scm

Problem description

Maven is a great tool, but it has one constraint that makes it inapplicable in many cases.
It can checkout the whole project (pom.xml and sources) from a single place in SCM storage using a single tag.

It's quite suitable in open-source projects, where new versions of projects don't appear too often.
But in our company we have a lot of components, each of which is stored in its own place in SCM and tagged with its own tag. To make using Maven possible, we need to create either BASH or ANT script which would check out all required projects from SCM.

This is the concern:

why do I need Maven if i still need ANT for running it?

If only Maven could do it by itself...

After thinking a while, we have created a solution.

Solution

As known, Maven SCM plugin has several SCM providers: CVS, Subversion, ClearCase etc.
Idea is to create one more provider, Proxy SCM. Its main goal is to checkout different modules from different places calling other SCM providers.
We need to extend pom.xml with 2 attributes:
  • /project/modules/module@scmProvider
  • /project/modules/module@scmHost
  • /project/modules/module@path
  • /project/modules/module@tag


Given pom.xml of form:

<project>
<scm>
<connection>scm:proxyscm:cvs:pserver:@myhost:dir:nodule</connection>
<developerConnection>scm:proxyscm:cvs:pserver:bla-bla</developerConnection>
</scm>
<modules>
<module path="robots/terminator/sdk">terminator_sdk</module>
<module path="robots/terminator/ui" tag="terminator_ui_v1-2">terminator_ui</module>
<module scmProvider="svn" scmHost="http://svn.apache.org/repos/asf/commons/proper/launcher" path="trunk">commons_launcher</module>
</modules>
</project>
,

The proxy-scm provider checks out all the 3 projects into 3 subfolders in current folder, resulting in the following content:
  • pom.xml
  • terminator_sdk
  • pom.xml
  • src
  • terminator_ui
  • pom.xml
  • src
  • commons_launcher
  • pom.xml
  • src


Implementation

The code of proxy-scm provider is quite simple: it just needs to parse file "pom.xml" and call appropriate SCM provider for every module.

I would like to commit this code into the next Maven SCM version (1.1). But it seems those guys don't respond to JRA issues neither commit patches into new version :(

12 комментариев:

Eugene Kuleshov комментирует...

As somewhat related to "those guys", I can tell you, that isn't terrible great idea to add random attributes to the Maven's pom, because it is very intrusive and such POM changes will affect many users.

Also, with Subversion you can map arbitrary svn repositories into a single tree, using svn:externals property. Though it seem like you are stuck with CVS.

So, if I were you, I'd wrote new Maven plugin and use custom plugin configuration handler to specify location of your modules in SCM and then delegate to the SCM manager to do the checkout. As a result you'd have plugin working with a standard Maven and will have full control on configuration.

asolntsev комментирует...

Well.. It seems to be more or less the same idea. With exception that I don't need to write any additional files, plugin configurations and give additional goals to the Continuous Integration server.
Just the standard pom.xml and standard "scm:checkout" goal which is by default performed by CI.

Why you think those POM changes will affect many users? They will not. Those attributes are optional - most of users will not use them and will not even know about them.

Eugene Kuleshov комментирует...

Compatibility is a strange thing and it has many faces. The short version is that changes like you are proposing unlikely get into Maven 2.0.x (that will require to chenge POM XSD and also the Maven model). It might get into 2.1, but I still doubt that it is a good idea to use <modules> element for stuff you've put in there.

Another thing is that maven-scm designed to be simple scm providers and should have no dependencies on the Maven POM.

Not sure what you meant about additional files or plugin configurations (the latter would be simply different place for the same config). Though you've made good point about different plugin and new goals. So it would seem a good idea to extend scm plugin and use its configuration to declare module locations.

<plugin>
<artifactId>maven-scm-plugin</artifactId>
<configuration>
<modules>
...
</modules>
</configuration>
</plugin>

asolntsev комментирует...

Yes, I like this idea too. Why not.
The only thing that I don't like is that I need to write <<modules> section at 2 places.
IMHO it would be nice to avoid any duplication and write things only once (that's Java slogan :) ).

Yes, we will need to change POM XSD - why not? I don't see why it's terrible.

Eugene Kuleshov комментирует...

Well, your modules structure is breaking default Maven's conventions and you are trying to glue it all back using some additional configuration instead of refactoring your modules in version control system. I haven't seen similar requests from other users, so plugin configuration seem like a reasonable compromise and less intrusive in this case. If it will became popular, that can be reviewed later for Maven 2.1...

asolntsev комментирует...

I agree.


PS. You haven't seen similar requests because people often don't know where to put such requests. When I first started using Maven, I could not find any suitable forum etc. And finally, I have heard from several companies' employees that they are writing ANT or BASH scripts.

Dennis M. комментирует...

@Eugene:

Q: When is a "default convention" not a default convention?

A: When it is effectively a rule mandated by poor build system design.

Eugene Kuleshov комментирует...

Eh?

Dennis M. комментирует...

My own investigation into this issue suggested that Maven cannot be configured or extended to work around it. Customisation requires creating one's own replacement versions of Maven components, or a non-Maven solution.

So referring to Maven's "default convention" around module structure is inaccurate. "Default convention" implies flexibility.

I'd love to be proved wrong here.

Eugene Kuleshov комментирует...

I still don't get it why it is such a big deal. Unless I am missing something, the issue that Andrei blogged about basically boils down that he wanted to be able to build several independent modules together. Basically all you need to create a plugin that would have list of those modules (standard GAV coordinates should be sufficient), so it would then get the pom for each module, grab SCM url for the project and then run checkout to a specified location and then it could simulate reactor build on those modules. Am I missing something or you are saying it is not possible to implement such plugin?

Dennis M. комментирует...

I do not feel that duplicating module info in plugin config is a desirable compromise. Config and/or custom plugins that eliminated the need to duplicate module info would be a desirable compromise. I appreciate your input though, and will consider the solution you suggest.

Bottom line, Maven should not make this so difficult. Difficulty is evidenced by people (I add myself to Andrei's list) choosing a non-Maven solution, as well as the following references:

Jason Van Zyl: "Trying to support multiple SCM entries or modules in a single POM would lead to a great deal of unwanted complexity. If you require that sort of thing I would say you some decomposition (sic) you could with that project."

http://markmail.org/message/f3jp2k4k4k3f2jvz

Further evidence of lack of plug-and-playability:

http://docs.codehaus.org/display/MAVEN/Substitution+of+Custom+Maven+Components

@Andrei, I'd be interested in knowing which path you've taken since your last post on this.

Eugene Kuleshov комментирует...

The custom plugin I described would replace modules section with its own configuration. So, there woun't be any duplication.