SNAPSHOT dependencies from Maven plugins are never checked for updates

I've put together a solution that works for Grails 1.3 for anyone needing this now.

https://github.com/alkemist/grails-snapshot-dependencies-fix

I am wondering whether your solution could be repurposed for Groovy too:
http://jira.codehaus.org/browse/GROOVY-3819

Paul: it could if we can find someway to make Ivy use a different class for the ?ibiblio? element in the config XML, or people would have to use a custom repo definition which isn't the end of the world.

Could this bug eventually cover the case of automatically updating a SNAPSHOT plugin?
I mean not only honouring plugin dependencies on other SNAPSHOT libraries, but also downloading the zip and replacing plugin contents into grails cache for obsolete snapshots.
That would be very useful for development of private plugins.

Cheers
Davide

That's possible, but outside the scope of this issue. It's covered by GRAILS-7172.

Ok, since this issue is in progress while GRAILS-7172 is still in Open state, it would be great if someone (Graeme?) could give also a look to the latter. Is there any chance?

Cheers
Davide

Even though GRAILS-7172 is closed, this still doesn't work (see my comment on that issue). Is there any workaround other than clearing the plugin cache from the .grails folder?

I am also seeing this problem in Grails 2.0.0.RC1

It works for me if the snapshot is published a maven compatible repository. I'm not sure about local snapshots. Maybe a separate issue.

@Jon: this is working for me. What repository is your snapshot in?

My mistake, my comment was not strictly about plugins. The effect I am seeing is that local-repository installed maven snapshots are not picked up. I have to remove them from the ivy cache.

I am having the same issue as Jon. Is there a separate JIRA item open for this already?

Jon / Andrew,

Can you please provide step by step instructions on how to reproduce the behaviour you are seeing and describe the behaviour you are expecting.

1) In a local Maven project with a version ending in -SNAPSHOT, use "mvn clean install" to build the project and drop it into your local .m2 directory.

2) Add the artifact as a dependency to a Grails project. Run the Grails project – the artifact will be pulled into your .grails Ivy cache.

3) Run "mvn clean install" again. This updates the artifact in your local .m2 directory.

4) Rebuild the Grails project. The artifact is not updated in the Ivy cache, and the project builds with the old version of the artifact. The only way to pull in the updated artifact is to first delete the old one from the Ivy cache.

Expected behavior is to check the mavenLocal() repository for changes each time. Maven does this by default for versions ending in -SNAPSHOT; not sure about release versions.

+1 for re-opening this, seeing the same thing as Andrew Todd with 2.0.4. Seems to be specific to local snapshot resolution. Spent half an hour trying to work out why it's not working but didn't get anywhere.

Is there any way to cause the host app to sync to an updated plugin (using grails 2.0.3)? Whether I use a SNAPSHOT version (0.2-SNAPSHOT), or upgrade the plugin to a new release version (0.3), nothing I do causes the plugin's artifacts to be updated in the host application. (I've got a controller and a few /src/groovy files that need updating.)

Here are the steps I followed:

1) After modifying the plugin and incrementing the version: clean, compile, package and maven-install it. Upon examining the local maven repo, the new version is there and looking good.

2) On the host app, run uninstall-plugin <plugin-name> <plugin-version>. Running list-plugins shows that the plugin is no longer installed. (Note that spring-security-core 1.2.7.3, upon which this plugin depends, is still installed.)

3) Also on the host app, upgrade the plugin version in BuildConfig.groovy, then run clean, compile and run-app. When the host app is starting up, my plugin descriptor prints its current version correctly to system.out.

However, none of the plugin's artifacts in the host application have been updated. The only way I have found to upgrade plugin artifacts within the host app is to manually move the modified artifacts into place.

John - have you tried with the --resolve-dependencies flag:

grails compile --resolve-dependencies

Graeme, yes - I tried both the --resolve-dependencies flag and the --refresh-dependencies flag (suggested elsewhere) to compile the host app (after cleaning), and neither one of them seems to make a difference. This is the case whether using a release-type version (0.5) for the plugin or a development-type version (0.6-SNAPSHOT). Additionally, I removed the plugin reference in the host app's BuildConfig.groovy (I was not doing this previously).

Here's an updated series of steps that I performed - un-installing and re-installing the plugin - but the artifacts never get refreshed:

grails list-plugins (plugin IS there)
grails uninstall-plugin <plugin-name>
REMOVE plugin reference from host's BuildConfig.groovy
grails compile --resolve-dependencies
grails clean
grails list-plugins (plugin NOT there)
ADD plugin reference back to host's BuildConfig.groovy
grails compile --resolve-dependencies
grails list-plugins (plugin IS there, referenced correctly at startup)
BUT artifacts have not been updated.

Here is my comment from issue GRAILS-7810 (The issues seem to be linked, this being a more appropriate issue for my comment)

I am using Grails 2.0.3.

I am having the same issue. I am trying to resolve a dependency via my local maven repository. This is not working.

I have also tried to explicitly state 'changing=true'

        compile('com.qualica:flexifin-common:1.0-SNAPSHOT'){
            changing = true
        }

I made changes to the dependency, and 'mvn install' it

The Ivy info log level summary output is

	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      compile     |  143  |   5   |   0   |   13  ||  130  |   0   |
	---------------------------------------------------------------------

I am using Grails 2.0.4, and having the same issue. Even when the file is found in my local maven repository, it uses the out of date version from ivy cache.

test ('com.peoplenet.pfm:domaintest:1.1.1-SNAPSHOT') {changing = true}

After updating the domaintest dependency, running a maven deploy and install on it, then running a grails test-app on the project with the dependency on domaintest:

localMavenResolver: no namespace defined: using system
don't use cache for com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT: changing=true
         trying C:\Users\tbeukhof/.m2/repository/com/peoplenet/pfm/domaintest/1.1.1-SNAPSHOT/domaintest-1.1.1-SNAPSHOT.pom
                tried C:\Users\tbeukhof/.m2/repository/com/peoplenet/pfm/domaintest/1.1.1-SNAPSHOT/domaintest-1.1.1-SNAPSHOT.pom
        localMavenResolver: found md file for com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT
                => C:\Users\tbeukhof\.m2\repository\com\peoplenet\pfm\domaintest\1.1.1-SNAPSHOT\domaintest-1.1.1-SNAPSHOT.pom (1.1.1-SNAPSHOT)
        parser = pom parser
No entry is found in the ModuleDescriptorCache : C:\Users\tbeukhof\.grails\ivy-cache\com.peoplenet.pfm\domaintest\ivy-1.1.1-SNAPSHOT.xml
post 1.3 ivy file: using exact as default matcher
        found ivy file in cache for com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT (resolved by cache): C:\Users\tbeukhof\.grails\ivy-cache\com.peoplenet.pfm\domaintest\ivy-1.1.1-SNAP
SHOT.xml
        default-cache: revision in cache (not updated): com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT
        checking com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT from cache against [none]
        module revision kept as first found: com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT from cache
        found com.peoplenet.pfm#domaintest;1.1.1-SNAPSHOT in cache

I'm experiencing this in 2.1.1 as well. My best guess for why its happening is that mavenLocal() is creating a new FileSystemResolver, whereas everything else is using the fixed SnapshotAwareM2Resolver. Perhaps a SnapshotAwareFilesystemResolver needs to be created?

Using latest master / 2.1.x branches if I do the following steps:

grails create-plugin foo-bar
cd foo-bar
grails create-controller foo
grails maven-install
cd ..
grails create-app myapp

Then modify BuildConfig:

compile ':foo-bar:0.3-SNAPSHOT'

Then run

grails compile

The plugin is installed. If I think modify FooController and the following in the plugin:

grails maven-install

Then run the following with the app:

grails compile

The resulting output is:

| Updating snapshot plugin 'foo-bar' with version '0.3-SNAPSHOT'
| Compiling 114 source files

And the snapshot is correctly updated from the local Maven cache. Can somebody who this issue is a problem for please posts steps on how to reproduce a failing snapshot update?

Thanks

The Maven dependencies in our application are purely Maven projects, so we can only run a maven install, not a grails maven-install (and we didn't use grails create-plugin to create them). Is that the difference?

shouldn't make a difference no, but if you provide steps to reproduce / attach an example that demonstrates how / when it is failing I can dig further. Thanks.

Graeme- I was able to reproduce with the following commands using Grails 2.1.1 on Max OSX 10.7.4. It was also verified by a coworker using Grails 2.0.3.

grails create-plugin foo-bar
cd foo-bar
grails create-controller foo
sed -i '' -E "s/version = \"0.1\"/version = \"0.3-SNAPSHOT\"/" FooBarGrailsPlugin.groovy
grails maven-install
cd ..
grails create-app myapp
sed -i '' -E "s/compile ':cache:1.0.0'/compile ':cache:1.0.0'; compile ':foo-bar:0.3-SNAPSHOT'/" myapp/grails-app/conf/BuildConfig.groovy
cd myapp
grails compile
grails compile #NOTE THE SECOND COMPILE
cd ../foo-bar
grails create-controller bar
grails maven-install
cd ../myapp
grails compile # does NOT detect the plugin snapshot change

The important step here is the second grails compile before updating the snapshot; if you omit it, the snapshot detection works properly as you indicated. Also, if you manually run grails refresh-dependencies, then it will detect the change.

@Ellery but this is by design, there is caching to avoid the performance cost of having to resolve dependencies every time, if you run:

grails compile --resolve-dependencies

Then the snapshot change would be detected

@Graeme Then my experience could just be user error- I did not realize that running compile with resolve-dependencies or calling grails refresh-dependencies manually was necessary to ensure that the dependencies are actually up to date. I only tend run grails compile explicitly in circumstances where I would also want to make sure my dependencies are correct; the fact that this works some but not all of the time may be why I assumed it was a bug. Would it make any sense to write something to the console indicating that the cache is being used in this way? In any event- I will make the appropriate adjustments and see if I still suffer from this issue. Thanks!

I am using an IDE (STS 2.9.2), with Grails 2.0.4.

• Create a Maven project called common with a pom.xml with version 1.0 and a Constants.java that has a few public static fields.
• Do a Maven install on common
• Create a Grails app myapp
• Add the dependency in BuildConfig.groovy (compile 'common:1.0-SNAPSHOT')
• Do a Grails Refresh Dependencies on myapp (which actually equates to a grails compile --non-interactive)
• Create a MyAppController.groovy in myapp and reference one of the constants in Constants.java.
• Add another constant to Constants.java
• Do another Maven install on common
• Do another Grails Refresh Dependencies (grails compile --non-interactive) on myapp
• Try to reference that new constant from MyAppController.groovy, and it won't have the updated version of Constants.java (the ivy-cache will still have the old version of common.jar)

I did try subbing out grails compile --non-interactive with both grails compile --resolve-dependencies and grails refresh-dependencies, but no luck.

@Tom - i will try reproduce with the latest master using the steps you provided

@Tom - so I'm not able to reproduce with your steps either. Although I used 'gradle install' rather than 'mvm install' not sure why that would make a difference. One thing to note, the argument is --refresh-dependencies not --resolve-dependencies

One other note, I tested with Grails 2.1.1

I tried upgrading to Grails 2.1.1, and using grails compile --refresh-dependencies, and it still uses the old version from ivy-cache instead of pulling the latest from .m2. Here is the verbose output when doing the grails compile --refresh-dependencies on the grails project vehicle-settings with the Maven dependency on vehicle-api.

== resolving dependencies org.grails.internal#vehicle-settings;1.1.16-SNAPSHOT->com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT [compile->default]
cache: Checking cache for: dependency: com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT {compile=[default]}
don't use cache for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT: changing=true
		tried C:\workspaces\pfm\pfm-trunk\ui\components\vehicle-settings/lib/vehicle-api-1.1.16-SNAPSHOT.jar
		tried C:\Users\tbeukhof\.grails\2.1.1\projects\vehicle-settings\plugins\hibernate-2.1.1/lib/vehicle-api-1.1.16-SNAPSHOT.jar
		tried C:\Users\tbeukhof\.grails\2.1.1\projects\vehicle-settings\plugins\release-1.0.0/lib/vehicle-api-1.1.16-SNAPSHOT.jar
		tried C:\Users\tbeukhof\.grails\2.1.1\projects\vehicle-settings\plugins\svn-1.0.1/lib/vehicle-api-1.1.16-SNAPSHOT.jar
		tried C:\Users\tbeukhof\.grails\2.1.1\projects\vehicle-settings\plugins\tomcat-2.1.1/lib/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsPlugins: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1/lib/com.peoplenet.pfm.core.component.vehicle/vehicle-api/ivy-1.1.16-SNAPSHOT.xml
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1/lib/com.peoplenet.pfm.core.component.vehicle/vehicle-api/jars/vehicle-api-1.1.16-SNAPSHOT.jar
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1/lib/com.peoplenet.pfm.core.component.vehicle/vehicle-api/bundles/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsHome: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1\src\libs/vehicle-api-1.1.16-SNAPSHOT.xml
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1\src\libs/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsHome: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1\dist/vehicle-api-1.1.16-SNAPSHOT.xml
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1\dist/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsHome: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
		tried C:\Users\tbeukhof\.grails\2.1.1\cached-installed-plugins/vehicle-api-1.1.16-SNAPSHOT.xml
		tried C:\Users\tbeukhof\.grails\2.1.1\cached-installed-plugins/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsHome: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
don't use cache for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT: checkModified=true
		tried C:\Users\tbeukhof\Downloads\springsource\grails-2.1.1/plugins/vehicle-api-1.1.16-SNAPSHOT.jar
	grailsHome: no ivy file nor artifact found for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
don't use cache for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT: changing=true
		tried C:\Users\tbeukhof/.m2/repository/com/peoplenet/pfm/core/component/vehicle/vehicle-api/1.1.16-SNAPSHOT/vehicle-api-1.1.16-SNAPSHOT.pom
	localMavenResolver: found md file for com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
		=> C:\Users\tbeukhof\.m2\repository\com\peoplenet\pfm\core\component\vehicle\vehicle-api\1.1.16-SNAPSHOT\vehicle-api-1.1.16-SNAPSHOT.pom (1.1.16-SNAPSHOT)
	default-cache: revision in cache (not updated): com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT
	found com.peoplenet.pfm.core.component.vehicle#vehicle-api;1.1.16-SNAPSHOT in cache

i suggest you attach an example project with steps to reproduce

Should that be -refresh-dependencies or --refresh-dependencies? I only use a single dash.

Ok, the files for the vehicle-api (Maven) and vehicle-settings (Grails) projects are attached. pom.xml and VehicleConstants.java go in their appropriate places under vehicle-api, and the rest go in their appropriate places under vehicle-settings. Here are steps to reproduce (I did try both --refresh-dependencies and -refresh-dependencies):

cd vehicle-api
mvn install
cd ../vehicle-settings
grails compile --refresh-dependencies
grails test-app # should run without error

Uncomment second constant in VehicleConstants.java under vehicle-api.
Uncomment second println in VehicleController.groovy under vehicle-settings.

cd ../vehicle-api
mvn install
cd ../vehicle-settings
grails compile --refresh-dependencies
grails test-app # errors

Fixed in 2.3 with Aether dependency management

Same issue - Grails doesn't update SNAPSHOT dependencies and uses cached value.

I don't understand how is it possible to have such stupid bug after 4-5 years of development? I'm reading http://grails.org/doc/latest/guide/conf.html#changingDependencies section, use this info and it doesn't work. How can I trust anything from documentation after that?

>Fixed in 2.3 with Aether dependency management
Ok. What about people that will use old dependency management?

Grails 2.2.0

Nothing we can do about it for older versions, complain about it to the Apache Ivy people (which is what Grails uses for dependency resolution)