MR: https://gitlab.wikimedia.org/repos/maven/wmf-jvm-parent-pom/-/merge_requests/27

@amastilovic is this a quick fix?

@TJones could you try to replicate this step by step from a fresh repository (cleanly fetched from gerrit and no target folders).
By default I think maven is supposed to ignore these folders when moving them to the target folders. I believe we should find what is causing them to appear in the first place because I'm afraid that fixing duplicate-finder-maven-plugin might be too late because it'd mean that these .DS_Store folders already have leaked into the build directories which could cause them to be published silently.
If on the other hand you found third-party jars (dependencies we use) with such folder in them this is a different story, we might possibly try to raise the issue to the lib owner and use the workaround you suggested to make our build pass (but haven't hit this problem myself so hopefully it's not the case).

I believe we should find what is causing them to appear in the first place

That would be me opening folders. The MacOS Finder creates .DS_Store files to record information about the folder—how it is sorted, the position of icons, where the window is located, etc. https://en.wikipedia.org/wiki/.DS_Store#Purpose_and_location :

The file .DS_Store is created in any directory (folder) accessed by the Finder application...

(There is also a "Problems" section... yeah, we know!)

I have a script that deletes all the .DS_Store files under a given directory, but sometimes the Finder is very aggressive about putting them back, and it happens so fast that I can't build between deleting them and them being restored by the OS. (This has varied over the years based on my operating system version and, I think, what OS jobs are running on my laptop at the time.)

I don't think .DS_Store files are likely to end up in our builds because they are usually listed in .git_ignore, so they don't get committed. As long as we don't build the published zips on a Mac, they won't be in the repo and won't end up in the zips. Then it's only a problem for local Mac dev builds, which in my case never leave my laptop.

A quick search on Gerrit shows that .DS_Store files do make it into repos, though.. especially in big commits, but also in some smaller ones. (Maybe what we need is a default .git_ignore for all new projects.)

So, if you are really worried about them somehow making their way into a build, we can abandon this task and people working on a Mac can add this kind of fix to their project pom locally, but not commit it, so that the presence of more than one .DS_Store file will still cause an error at build time—I think the error requires there to be more than one... but they are like rats.. if you see one, there are almost always a lot more you didn't see.

In T407514#11319096, @TJones wrote:

I believe we should find what is causing them to appear in the first place

That would be me opening folders. The MacOS Finder creates .DS_Store files to record information about the folder—how it is sorted, the position of icons, where the window is located, etc. https://en.wikipedia.org/wiki/.DS_Store#Purpose_and_location :

Thanks, by chance do you remember if this happens only when opening a target folder, if you have a workflow that relies on opening some of the target folders I'm not sure what we could do, if you never open those there is perhaps something we could do to avoid them early?

Overall I'm not too worried about pushing them to gerrit because as you said they're generally .gitignored. I'm more concerned about them leaking into the jars we publish, someone with a mac might still want to release some of our plugins, but perhaps I'm too paranoid here, the jar plugin seems to also ignore them (so that's a second protection besides the filter that's supposed to prevent them from appearing in the target folder) and the release plugin is supposed to fetch a clean checkout in the target folder prior to building the jars so that would be a third protection.

So perhaps the workaround is fine if there's nothing we can do to prevent them from proliferating into unexpected folders, but I'm not a mac owner so it's hard for me to test things out.

I often open the target folder to find the jar so I can deploy it locally. I also use the Finder with Java projects to recursively open folders to get all the way to the bottom quickly. (Anything that relies on people not opening folders is doomed to fail eventually.)

Is there a command or config that can exclude certain files from being included when building jars or plugin zip files? I've looked, but haven't found anything.

In T407514#11320956, @TJones wrote:

I often open the target folder to find the jar so I can deploy it locally. I also use the Finder with Java projects to recursively open folders to get all the way to the bottom quickly. (Anything that relies on people not opening folders is doomed to fail eventually.)

thanks, this is mainly what I was curious about.

Is there a command or config that can exclude certain files from being included when building jars or plugin zip files? I've looked, but haven't found anything.

The jar plugin should filter those, I tried hard to make them appear in the jar, source jars & javadoc but failed, I suppose we are fine?

I believe we have two options to make this less annoying:

• exclude .DS_Store from this duplicate-finder plugin in the parent pom, but we'd have to re-add it for projects that override the default settings
• run your build with ./mvnw verify -Dduplicate-finder.skip=true that will explicitly skip the whole plugin locally or perhaps even better with ./mvnw verify -Dduplicate-finder.preferLocal=false which should tell the plugin to use generated jars instead of local folders.

I have a slight preference for the latter but I understand that we may want the former if this is something that mac users are generally annoyed with.

Pinging @pfischer which I believe is running mac and might have opinions on this.

In T407514#11322205, @dcausse wrote:

• exclude .DS_Store from this duplicate-finder plugin in the parent pom, but we'd have to re-add it for projects that override the default settings

Couldn't the child pom use combine.child as described in the doc to not need to re-declare? This might not be obvious to child projets, so it might need some documentation. (I haven't checked the actual use case, so I might be missing something).

In general, the approach we've tried to take was to make projects as easy to build as possible. You should be able to checkout a project and run ./mvnw clean verify and have it work. Needing to add specific -D arguments breaks that rule.

In T407514#11322446, @Gehel wrote:

In T407514#11322205, @dcausse wrote:

• exclude .DS_Store from this duplicate-finder plugin in the parent pom, but we'd have to re-add it for projects that override the default settings

Couldn't the child pom use combine.child as described in the doc to not need to re-declare? This might not be obvious to child projets, so it might need some documentation. (I haven't checked the actual use case, so I might be missing something).

In general, the approach we've tried to take was to make projects as easy to build as possible. You should be able to checkout a project and run ./mvnw clean verify and have it work. Needing to add specific -D arguments breaks that rule.

Perhaps we could set this in the parent pom:

<plugin>
	<groupId>org.basepom.maven</groupId>
	<artifactId>duplicate-finder-maven-plugin</artifactId>
	<configuration>
                <preferLocal>false</preferLocal>
	</configuration>
</plugin>

my understanding is that we always package the jar prior to running the check, and adding this new config option should properly propagate to children poms (unlike array entries).

<preferLocal>false</preferLocal> does not seem to work, as tested on @TJones Mac laptop.

Change is merged, we need to publish a new release of the parent pom.

Done, 1.98 is released (externally and internally)