Follow changes in your repositories
If you are a Linux (or UNIX) user and using SCM (Source Control Management) software like Fossil and Git, maybe like me you do not want to immediately commit your changes. You do not want drafts to be tracked. But from time to time you forget about those uncommitted changes and risk to lose your work.
Would you like to see a list of your repositories with uncommitted changes every time you open a new shell session or a terminal? To have something like this?
Uncommitted changes present in these directories:
/home/nales/git-project
/home/nales/fossil-project
/home/nales/website
~ $
This article is about a set of scripts I daily use to automatically follow the changes in my repositories. You can find my scripts (and others) in my scripts repository. Those scripts work on Linux, they may work on BSD systems or MacOS. The goal of this article is not to explain line-by-line the scripts, but to give you the idea of using a similar solution and to show you how to adapt my solution into yours.
The term SCM
in this article refers to Source Control Management software
like Fossil or Git. If you do not know what is an SCM, I highly recommend you to check out Git which is the most popular one right now in 2020 (even if I prefer Fossil). In a nutshell, an SCM is a software used to organize changes in your files (usually code source). You can diverge from the current state to commit changes and apply them later, you can collaborate with other people and merge their changes to yours, you can also check out the history of changes to see how your files evolved and revert changes.
Planning the solution
The goal is to check out if there are uncommitted changes from our side in the repositories we follow. The goal is not to check out distant changes from other people. We use Fossil and Git.
There will be two scripts for that. The first script, which-scm
, is used to find which SCM is used in a directory and what is the root path of the repository. The second script, follow-scm
, is used to add the repositories we want to follow and to check out uncommitted changes. When we open a new shell in a terminal, the repositories with uncommitted changes are displayed.
Finding the SCM with which-scm
Both Fossil and Git create an hidden file in the root path of their repositories. Fossil creates a SQLite database name .fslckout
. (Fossil was created by the developers of SQLite.) Git creates a directory named .git
.
We want which-scm
to print in the standard output the name of the SCM and the root path of the repository separated by a tabulation to simplify integration in other scripts. If an SCM is found, the script returns 0
. If no SCM are found, because we are not in a repository or in another SCM repository (SVN for example), nothing is printed and the script returns 1
.
#!/bin/sh
print_info_and_exit() {
printf "%s\t%s\n" "$1" "$(pwd)"
exit 0
}
check_local() {
if [ -d ".git" ]; then
print_info_and_exit "git"
elif [ -f ".fslckout" ]; then
print_info_and_exit "fossil"
fi;
}
# If no arguments given, takes the current directory as an argument.
path="${1:-$(pwd)}"
if [ ! -d "$path" ]; then
printf "%s is a not a directory path.\n" "$path" 1>&2
exit 2
fi
# Checking the directory and its parents until giving up at "/" path.
cd "$path"
while [ "$(pwd)" != "/" ]; do
check_local
cd ..
done
exit 1
Using this script in a shell gives us those results.
$ which-scm /home/nales/git-project/src/
git /home/nales/git-project
$ which-scm /home/nales/fossil-project/src/
fossil /home/nales/fossil-project
$ which-scm /home/nales/not-a-repository/
$ echo $?
1
Writing follow-scm
follow-scm
will be used to choose which repositories in the system we want to follow. It will use which-scm
to keep track of the SCM used and the root repository path. Let's write it now.
Defining the files and script commands
We need to find where to store the list of repositories follow-scm
will follow. In order to let the user organize where the files will be in its system, the script will use the XDG Base Directory specification. We write our data in the XDG_DATA_HOME
directory in a follow-scm
file.
There will be a bunch of commands to define. I prefer to use variables to store the commands, it makes refactoring easier. I will not implement in this article every command because this is not the goal. See my scripts repository for a full implementation. Here is the source code skeleton.
#!/bin/sh
COMMAND_HELP="help"
COMMAND_ADD="add"
COMMAND_REMOVE="remove"
COMMAND_AUTOREMOVE="autoremove"
COMMAND_CHECK="check"
COMMAND_LIST="list"
COMMAND_EDIT="edit"
DATA_FILE="${XDG_DATA_HOME:-$HOME/.local/share}/follow-scm"
if [ ! -e "$DATA_FILE" ]; then
touch "$DATA_FILE"
fi
# We will need a temporary file for remove and autoremove commands
# WARNING: Linux uses /tmp but I am not sure for other OSes. Adapt the script!
TEMP_DIR="${TMPDIR:-/tmp}"
TEMP_FILE="$TEMP_DIR/follow-scm-$$"
is_known_scm() {
[ "$1" = "fossil" ] ||
[ "$1" = "git" ]
}
# We will define the commands here.
if [ $# -eq 0 ]; then
help
else
case "$1" in
"$COMMAND_HELP")
shift; help "$@"
;;
"$COMMAND_ADD")
shift; add "$@"
;;
"$COMMAND_REMOVE")
shift; remove "$@"
;;
"$COMMAND_AUTOREMOVE")
shift; autoremove
;;
"$COMMAND_LIST")
shift; list
;;
"$COMMAND_EDIT")
shift; edit
;;
"$COMMAND_CHECK")
shift; check
;;
*)
print_unknown_command "$1"
esac
fi
Adding repositories to follow
The add
takes a path as an argument. If there are no paths given, follow-scm
uses the current directory. We simply store the result of which-scm
in the data file.
add() {
path="${1:-$(pwd)}"
which_output="$(which-scm "$path")"
if [ -z "$which_output" ]; then
printf "%s is not in a SCM repository.\n" "$path"
exit 1
fi
scm="$(printf %s "$which_output" | cut -f1)"
if ! is_known_scm "$scm"; then
# Should print error and instructions for help here
exit 1
fi
# No need to add again an already present entry
if grep --quiet "^$which_output\$" "$DATA_FILE"; then
return
fi
printf "%s\n" "$which_output" >> "$DATA_FILE"
}
Then just add the repositories you want to follow.
$ follow-scm add /home/nales/git-project/src/
$ follow-scm add /home/nales/fossil-project/src/
Checking changes in Fossil and Git
With Fossil, we need two commands to check changes. The command fossil changes
display changes in the already tracked files, the command fossil extras
display new files that are not already tracked. If no changes are present, nothing is displayed.
With Git, we only need git status -s
to display changes. If no changes are present, nothing is displayed.
The check
command reads the data file filled by the add
command, and check one by one each repository. It prints the repositories with uncommitted changes.
get_scm_from_entry() {
printf "%s" "$1" | cut -f1
}
get_path_from_entry() {
printf "%s" "$1" | cut -f2
}
check() {
line_number=0
while read -r line; do
line_number=$((line_number + 1))
# Here tests should be present to check the integrity of the entries
# stored in the data file. To reduce the code, I skip them.
path="$(get_path_from_entry "$line")"
scm="$(get_scm_from_entry "$line")"
cd "$path" || continue
if [ "$scm" = "git" ] && [ -n "$(git status -s)" ]; then
printf "%s\n" "$path"
elif [ "$scm" = "fossil" ] && [ -n "$(fossil changes && fossil extras)" ]; then
printf "%s\n" "$path"
fi
done < "$DATA_FILE"
}
Now we can check the repositories with uncommitted changes:
$ follow-scm check
/home/nales/fossil-project
Removing entries
Without going into details here, to remove some repositories in the data file you can use something like that:
remove() {
path="${1:-$(pwd)}"
which_output="$(which-scm "$path")"
if [ -z "$which_output" ]; then
# Print here the error and suggest to use autoremove command
exit 1
fi
grep -v "^$which_output\$" "$DATA_FILE" > "$TEMP_FILE"
mv -f "$TEMP_FILE" "$DATA_FILE"
}
autoremove() {
printf "" > "$TEMP_FILE"
while read -r line; do
# ...
# Test here each line, if the entry is incorrect: skip it.
# ...
# Only write correct entries
printf "%s\n" "$line" >> "$TEMP_FILE"
done < "$DATA_FILE"
mv -f "$TEMP_FILE" "$DATA_FILE"
}
To remove the currently followed Git repository from the list of followed repositories, do:
$ follow-scm remove /home/nales/git-project/src/
I will not show other commands like help
or list
because it is not the goal here.
Displaying the list in new shell sessions
We want to display the repositories with uncommitted changes when opening a new session in a shell (when you open a terminal for example). To do that, add in your .bashrc
or your .zshrc
(or any equivalent file in your system) those lines:
# Print in red the list of followed repositories with uncommitted changes.
echo -en "\033[1;31m"
follow-scm check
echo -en "\033[0m"
It works, but this is not efficient. Every time you open a new shell session, you have to wait the end of the command before doing anything. Let's improve that.
The first thing we do is to create a script called follow-scm-check
which will print the result of follow-scm check
in a cache file, or delete this file if there are no results.
#!/bin/sh
# If called as a cron task, do not forget to export PATH environment variable
# so the path for this script and its dependencies (follow-scm, git, fossil...)
# are present. If you do not, maybe nothing will work because the shell used by
# cron cannot find the scripts.
CACHE_FILE="${XDG_CACHE_HOME:-$HOME/.cache}/follow-scm-check"
result="$(follow-scm check)"
if [ -n "$result" ]; then
echo "Uncommitted changes present in these directories:" > "$CACHE_FILE"
printf "%s\n" "$result" >> "$CACHE_FILE"
elif [ -f "$CACHE_FILE" ]; then
rm "$CACHE_FILE"
fi
Now we will call this script every 10 minutes using a cron task. If you do not know what is a cron task, it is a task that is scheduled according to the rules you set. Here is the cron task in my system, use the crontab -e
command to add it:
*/10 * * * * export PATH=$PATH":$HOME/bin:$HOME/.local/bin:/usr/local/bin/"; follow-scm-check
Frankly cron tasks are a real pain to work with. They do not execute into your usual shell so even if you script works when you execute it manually, maybe it does not work when executed as a cron task. You musk check the PATH
environment configuration. You want to see the error logs when there is a problem? Silly you! Do you really think that on Linux there is a standard way across all different Linux flavors for this kind of simple things? No wonder why $CURRENT_YEAR
is still not the year of the Linux desktop...
The explanation of the cron task above.
- We set up the task to be run every 10 minutes with
*/10 * * * *
. - The rest of the line is about the script to execute. The set up for
PATH
variable correspond of my needs on my system.- I saved
follow-scm-check
at$HOME/bin/
because it is a personal script. - I have my
fossil
executable at$HOME/.local/bin/
which is a standard directory to put executables. - And the scripts from my scripts repository which include
fossil-scm
are installed in/usr/local/bin/
as does suckless for their software. (You should check out theMakefile
in my repository in case changes were made since I wrote this article.)
- I saved
- Finally we can call
follow-scm-check
.
Now we can update the code in the .bashrc
or .zshrc
(or any equivalent) to print the uncommitted repositories.
# Print in red the list of followed repositories with uncommitted changes.
if [ -f ~/.cache/follow-scm-check ]; then
echo -en "\033[1;31m"
cat ~/.cache/follow-scm-check
echo -en "\033[0m"
fi
Tip: following every repository in your system
Instead of manually searching and adding all the repositories to follow in your system, you can use find
to automatize that.
To find the repositories, we can search for the .git
directory and the .fslckout
file. See the manual of find
in the OPERATORS
section for more information. We assume the repositories are in the directory Projects
.
$ find Projects \( -type d -a -name '.git' -o -type f -a -name '.fslckout' \)
Projects/Website/.fslckout
Projects/Scripts/.git
Projects/G-Code-Viewer/.git
find
has a command to execute a script for each thing it finds. We cannot use directly follow-scm add
because it works only on directories paths but here find
also returns .fslckout
paths. The simplest solution is to create a temporary script.
#!/bin/sh
# ${1%/*} removes the smallest suffix in "$1" with the pattern "/*". Here it
# removes from the argument "/.git" and "/.fslckout" so it returns the
# directory path.
# For example: "/home/nales/project/.git" becomes "/home/nales/project"
follow-scm add "${1%/*}"
We can now mix together our find
command with our temporary script.
$ find Projects \( -type d -a -name '.git' -o -type f -a -name '.fslckout' \) -exec ./add-scm {} \;
All the repositories are now followed. If you want to remove some, you can use follow-scm edit
which opens follow-scm
data file in a text editor. Here is the code of this command if you are curious:
edit() {
# Check VISUAL then EDITOR environment variable else choose vi
editor="${VISUAL:-${EDITOR:-vi}}"
"$editor" "$DATA_FILE"
}
In summary
To display all repositories in your system with uncommitted changes, you can use follow-scm
present in my scripts repository or create your own version. After that, simply create a task cron to regularly make a check. Print the check result when you open a shell session.
With all the explanation I gave you above, you are more than ready to make your own version. Have fun!