Git Conversion Lazarus

From Lazarus wiki
Revision as of 14:09, 1 July 2021 by Martin (talk | contribs) (Updating tags)

This page has the detailed steps used to convert the Lazarus SVN repro to git.


SubGIT

The initial conversion will be done using TMate Subgit.

Install

Unpack subgit to

   ~/subgit/subgit-3.3.11

Required

An author.txt file

 svn-name - git-name <email>

Run / main repro

   cd ~/subgit/subgit-3.3.11
   ./bin/subgit configure https://svn.freepascal.org/svn/lazarus laz.git
   
   gedit laz.git/subgit/config 

And add the following (the first line, replaces an existing setting)

   [svn]
     trunk = trunk:refs/heads/main
     gitCommitMessage = %message\\n\\n SVN %revision@%branch\\n
     
   [translate]  
     eols = false
     otherProperties = false
     ignores = false

The translate options prevent .gitignore and .gitattributes. Those can be translated, but will later be replaced...

   cp authors.txt laz.git/subgit/authors.txt
   ./bin/subgit install laz.git

Without skipping the gitignore/gitattributes, there will be an error at revision 47330. The process will hang. nterrupt it and editI

   lazarus.git/svn/.metadata

and set branches-maxRev and tags-maxRev to 47332 (it should be at 47330) https://stackoverflow.com/questions/18918215/skip-revisions-when-installing-subgit-repository Continue with

   ./bin/subgit install laz-bin.git

Stop the daemon

   ./bin/subgit shutdown laz-bin.git/

Run / binaries repro

   ./bin/subgit configure https://svn.freepascal.org/svn/lazarus laz-bin.git
   gedit laz.git/subgit/config 
     [svn]
     gitCommitMessage = %message\\n\\n SVN %revision@%branch\\n
     trunk = binaries:refs/heads/main
     cp authors.txt laz-bin.git/subgit/authors.txt

follow the steps above

Part 2

  • Updating gitignore
  • Prefixing all issue references with #
  • translating svn revisions to git hashes

Set a git user

   git config --local user.email "lazarus@lazarus-ide.org"
   git config --local user.name "lazarus"

Clean out left overs from subgit

   git for-each-ref --format="%(refname)" refs/svn/ | xargs -n 1 git update-ref -d

Clean left overs from binaries

There are a few tags, that lead to binaries (gdb.exe) ending up in the data

  git tag -d binaries_0_9_26
  git tag -d binaries_0_9_26_2
  git tag -d binaries_0_9_28
  git tag -d binaries_0_9_30
  git tag -d binaries_0_9_30_2
  git tag -d binaries_0_9_30_2RC1
  git tag -d binaries_1_0

Updating gitignore/gitattributes / Prefix issues with #

The below combines the 2 steps. It can be run in 2 separate calls.

It will run for about one hour.

   FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --index-filter '  cp ~/subgit/subgit-3.3.11/.gitignore  ~/subgit/subgit-3.3.11/.gitattributes . ; git add .gitignore .gitattributes '  --msg-filter ' perl -ne "while (s/((?:iss?ues?|[ (](?:bugs?|mantis|fix|fixes|patch)\b)(?: +id)?[ :]+(?:#?0*[1-9][0-9]{2,4}(?: *(?:[ ,]| and ) *))*)(0*[1-9][0-9]{2,4})\b/\1#\2/i) {}; print" ' -f  -- --all


Cleanup

   git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
   git reflog expire --expire=now --all ;git gc --prune=now


Translating references to svn rev (r12345) to #hash

This must be the very last filter-branch. As each filter-branch creates new hashes for the commits, and that would invalidate the hash refs created here.


Updating tags

This must be done after all filter-branches

  • Annotate all tags.
  • Create some more tags

This needs the git user being set

   git for-each-ref --format="%(refname:short)" refs/tags/ | xargs -L1 /bin/sh -c 'git tag -f -a $0 -m $0 $0 '


   git rev-list --in-commit-order  --reverse   fixes_2_2..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-2_3" -m "trunk to version 2.3" $0'
   git rev-list --in-commit-order  --reverse   fixes_2_0..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-2_1" -m "trunk to version 2.1" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_8..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-1_9" -m "trunk to version 1.9" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_6..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-1_7" -m "trunk to version 1.7" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_4..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-1_5" -m "trunk to version 1.5" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_2..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-1_3" -m "trunk to version 1.3" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_0..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-1_1" -m "trunk to version 1.1" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_30..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-0_9_31" -m "trunk to version 0.9.31" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_28..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-0_9_29" -m "trunk to version 0.9.29" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_26..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-0_9_27" -m "trunk to version 0.9.27" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_24..trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-0_9_25" -m "trunk to version 0.9.25" $0'
   git rev-list --in-commit-order  --reverse   trunk | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "trunk-0_0" -m "trunk init" $0'
   
   git rev-list --in-commit-order  --reverse   trunk..fixes_2_2 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-2_2" -m "fixes 2.2 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_2_0 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-2_0" -m "fixes 2.0 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_1_8 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-1_8" -m "fixes 1.8 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_1_6 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-1_6" -m "fixes 1.6 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_1_4 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-1_4" -m "fixes 1.4 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_1_2 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-1_2" -m "fixes 1.2 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_1_0 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-1_0" -m "fixes 1.0 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_0_9_30 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-0_9_30" -m "fixes 0.9.30 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_0_9_28 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-0_9_28" -m "fixes 0.9.28 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_0_9_26 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-0_9_26" -m "fixes 0.9.26 branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..fixes_0_9_24 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "fixes-0_9_24" -m "fixes 0.9.24 branch" $0'
   
   git rev-list --in-commit-order  --reverse   trunk..free-sparta | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "free-sparta" -m "free-sparta branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..gtk-splitup | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "gtk-splitup" -m "gtk-splitup branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..lcl-smartlink | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "lcl-smartlink" -m "lcl-smartlink branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..listviewsortindicator | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "listviewsortindicator" -m "listviewsortindicator branch" $0'
   git rev-list --in-commit-order  --reverse   trunk..macosfullkeyboardaccess | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "macosfullkeyboardaccess" -m "macosfullkeyboardaccess branch" $0'

Comparing data before/after filter-branch

With subject line / Will show changes in subject

   git log --pretty='format:%ae %an %cd  %s %d' --graph --all > ../log.txt

Without subject line / Should have no diff

   git log --pretty='format:%ae %an %cd %d' --graph --all > ../log.txt


Make sure you do that after "clean up" in each step.

Also "removing the binaries" introduces a diff.

Data

Please mail me for any changes. Do not make changes here. Otherwise the changes will not make it into the conversion.

.gitignore

   **/*.o
   **/*.ppu
   **/*.exe
   **/*.bak
   **/*.diff
   **/*.patch
   **/units/
   **/lib/
   **/*;[1-9]
   **/*.~*
   **/*.*~
   **/*.fpcunit.ini
   **/*.lps
   **/*.compiled
   /docs/chm/**/*.chm
   /docs/chm/**/*.xct
   /docs/chm/**/*.txt
   /libeay32.dll
   /ssleay32.dll
   /lazarus.cfg
   ide/revision.inc
   
   !**/patches/*.patch
   !**/patches/*.diff

.gitattributes

   * text=auto !eol
   **/*.pp text
   **/*.pas text
   **/*.inc text
   **/*.lfm text
   **/*.lrs text
   **/*.lps text
   **/*.lpr text
   **/*.lpi text
   **/*.lpk text
   **/*.lpl text
   **/*.lrj text
   **/*.dpr text
   **/*.dfm text
   **/*.pot text
   **/*.po text
   **/*.rc text
   **/*.txt text
   **/*.txt.sample text
   **/*.ini text
   **/*.cfg text
   **/*.iss text
   **/*.isl text
   **/*.xml text
   **/*.xsl text
   **/*.html text
   **/*.css text
   **/*.patch text
   **/*.md text
   **/*.bat text
   **/*.sh text
   **/*.compiled text
   **/*.bdsproj text
   **/*.plist text
   **/*.h text
   **/*.c text
   **/*.cpp text
   **/*.in text
   **/Makefile text
   **/Makefile.compiled text
   **/Makefile.fpc text
   **/*.exe -text
   **/*.dll -text
   **/*.so -text
   **/*.gif -text
   **/*.jpg -text
   **/*.ico -text
   **/*.svg -text
   **/*.bmp -text
   **/*.res -text
   **/*.ttf -text
   **/*.xpm -text
   **/*.cur -text
   **/*.db3 -text
   **/*.dbf -text
   **/*.mbf -text
   **/*.mdx -text
   **/*.fbk -text
   **/*.odt -text
   **/*.ods -text
   **/*.odg -text
   **/*.odp -text
   **/*.xls -text
   **/*.pdf -text
   **/*.zip -text
   **/*.rar -text
   **/*.tar -text
   **/*.tgz -text
   **/*.gz -text
   **/*.icns -text
   **/*.dcr -text
   components/aggpas/**/*.ppm -text
   **/*.app/Contents/MacOS/* -text