Thursday, July 30, 2020

Fixing Git after Moving a File Type to LFS

We ran into a problem a while back where we had a Git repo that had been in use for a while and then we decided one of the common file types should be in the LFS.  The obvious way to do this was to update the filters in the .gitattributes file in the root:

*.ext filter=lfs diff=lfs merge=lfs

However, after doing this, the various Git clients (command line, Source Tree, etc.) didn't actually move all the *.ext files into the LFS.  Any new commits we made that touched those files would automatically move them, but there were plenty of files that didn't get updated.  We noticed this because Git would notice some of these other files (even though they hadn't been modified) and give us the dreaded "Encountered X file(s) that should have been pointers, but weren't" error whenever doing Git operations.

A little Goolging led us to this method of getting the rest of the files into the LFS:

git lfs migrate import --no-rewrite path/to/file.ext

This seemed pretty straightforward.  We wrote a Python script that did an os.walk() across our project and compared it against the results of git lfs ls-files to see which of the new files had yet to be added to the LFS.  This came up with a couple hundred files for us.  We had the script also run the import command above and with that we had the remaining files where they needed to be.  Push to remote, and we're set.

We were not set.

When we attempted to view the Git repo in Source Tree and also Git Fork, the program either crashed or hung spinning on a load.  Somehow we had broken the Git repo.  We reviewed the logs in the command line client and noticed that the timestamps on all the import commands were December 31st, 1969.  Effectively a zero value in the date/time field for the commit, which seemed to confuse the graphical Git clients.  Back to Googling.

This StackOverflow answer from Ortomala Lokni provided a potential solution.  We needed to modify the process a bit for the Windows machines we were on.  Basically, the date setting needed to be two commands: set the committer date as an environment variable once:

set GIT_COMMITTER_DATE="2020-07-19T00:00:00"

Then we could proceed with the individual commit updates during the rebase:

git commit --amend --date="2020-07-19T00:00:00"

Of course, a major inconvenience here is that we still have the .gitattributes filter set to *.ext files should be in the LFS.  And when we do the rebase process, it takes the files out of the LFS which goes against the filter specification, and thus we can't amend the commit because of the filter error.  Luckily, Git has the ability to override the .gitattributes list temporarily by placing a file called "attributes" (no extension) in the ".git/info" folder in the project.  In that file we place the override entry:

*.ext !filter !diff !merge

This effectively makes Git ignore that file's LFS filter so the rebase can proceed without errors.  However, once the rebase was complete, we had to remove that override file again before resuming normal operations.

No comments:

Post a Comment