Tags: git »»»» Gitlab »»»» Software Development »»»» Web Development Tools
Github doesn't have to be the only game for git servers - while they run an excellent service, you can't install it on your own server, making it little better (in "Freedom" regards) than a closed source proprietary software package. Yes it's expedient to host your repositories on Github, but do we all have to cede this functionality to them? There happen to be several alternatives to github that offer a competent web-based git repository service. I've looked at Gitlab and now Gogs (Go Git Server) and while both are competent systems, I think I'll be running Gogs (in Docker) full time on my laptop. While both Gitlab and Gogs can run in Docker containers making it easy to run them full time in the background, Gogs is lighter weight.
It's useful first to explore WHY - why spend ones precious time on this planet doing such a task? You can easily sign up for a Github or Gitlab or Bitbucket account and just use their hosted service. Why, then, do it on your laptop?
In my case I'm setting up a git-based workflow for website and electronic book development and deployment. Meaning, I have software tools (
AkashaCMS) running in Node.js that build static HTML websites or electronic books (EPUB3 format). It's turning out to be nice to have a build system where I edit the files on my laptop, then commit edited files to a git repository somewhere, and the website is rebuilt/redeployed automagically for me.
I've set this up with Gitlab using the free Gitlab CI service. It's nice - you edit, push the changes, automatically Gitlab starts a build job, my software runs, and a couple minutes later the website is updated. I can do it all on my laptop, of course, but I've also convinced my girlfriend to use the software and she's not very technical. Meaning going to a command line and typing npm run build && npm run deploy is foreign. It's a lot simpler to tell her to use the website, edit the files through the repository web interface, then configure the system so it builds/deploys automatically in the background.
As the name ("Go Git Server") implies,
Gogs is a git server written in Go. While I'm sure it was nice for them to write the thing in a modern language, it makes it difficult to deploy to my VPS. Dreamhost in its infinite wisdom no longer allows VPS "owners" to perform commands as root nor to run background processes of our own. Sigh. It means I cannot simply run Gogs on my VPS and be done with it, at least not without renting another VPS from another service.
But with Docker on ones laptop it's easy to run Gogs on the laptop. I've also ordered a Raspberry Pi that I intend to use for running Gogs. The steps shown below should work for either. My laptop is of course a Mac OS X based - 2011 Macbook Pro - system. I have installed the Docker for Mac application - which is the best way to run Docker on your laptop, either Windows or Mac.
Finding the documentation for running Gogs under Docker is not intuitively obvious - go here:-
https://github.com/gogits/gogs/tree/master/docker
$ docker pull gogs/gogs
$ mkdir ~/gogs
$ docker run --name=gogs -p 10022:22 -p 10080:3000 -v $HOME/gogs:/data gogs/gogs
This is a little different than they recommend - namely, I'm putting the Gogs data volume in my home directory. The first command pulls down the Gogs image, and the second command makes a directory where Gogs will store its data. The third command has a lot going on, so let's examine it part by part.
With docker run you initialize an image as a container and execute that container. For an already initialized container, you use docker start. The --name argument gives it a nicer name than gogs/gogs. The two -p options map ports inside the running container to ports you access on the laptop's environment. The first maps the standard SSH port to port 10022, so any SSH interaction (such as interacting with a git repository) is done via that port. The second maps the Gogs web service to port 10080. The -v is what maps the local directory, $HOME/gogs, into the Gogs container as a volume, and it will be available as /data inside the container. The last argument specifies that we're running the gogs/gogs image.
When started you can visit
http://localhost:10080 and then go through an installation process. Then within a couple minutes you've got a web service running on your laptop that looks astonishingly like Github. (fewer features, though)
Once you've got it set up, it's easy to create a repository, to push repositories into the Gogs server and so forth. You do have to be careful of one thing, the port numbers Gogs tells you to access. Because Gogs does not know the web port number has been mapped to port 10080, it will tell you to access things at
http://localhost:3000 ... you'll have to edit that URL to use port 10080. Likewise it might give you an SSH URL for accessing a repository, and you'll have to insert port 10022 into that URL.
By the way, in the terminal window where you started Gogs a running log of your activity is being printed. While interesting, it's not how we want this to be over the long-term. First, kill the running Docker instance - by typing Control-C once or twice. Then run this:
$ docker start gogs
You can now pick up where you left off and your terminal is now free for doing other things.
SSH access from inside the Docker container - for autopushing the repository elsewhere
You probably do not want the repository to solely live on your laptop, right? Why not automatically push the repository to one on a server that has routine backups? Oh, and you should probably have backups happening for your laptop. Mac OS X makes this painless using the Time Capsule feature. I have a Drobo sitting on my home network, and Drobo has built-in capability to offer itself as a Time Capsule. I don't even have to think about it, the laptop takes care of backing itself up to the Drobo.
In my case the repository needs to be pushed to my VPS, and the build will occur there.
I was unable to find documentation for the following - but it does work fine.
$ docker exec -i -t gogs bash
The "docker exec" command runs a process inside a Docker container, while the -i -t flags assign a terminal (-t) and makes it run interactively (-i), and since we're executing bash that gives us a bash shell with which to run commands inside the container. That lets us explore the container and see what's what.
bash-4.3# ps auxf
PID USER TIME COMMAND
1 root 0:00 /bin/s6-svscan /app/gogs/docker/s6/
20 root 0:00 s6-supervise crond
21 root 0:00 s6-supervise openssh
22 root 0:00 s6-supervise syslogd
23 root 0:00 s6-supervise gogs
24 root 0:00 /usr/sbin/sshd -D -f /app/gogs/docker/sshd_config
25 root 0:00 /sbin/syslogd -nS -O-
27 git 0:13 /app/gogs/gogs web
168 root 0:00 bash
173 root 0:00 bash
179 root 0:00 ps auxf
That's the apps running, and notice that openssh is running. It means we can do the following:
# cd /data/git
bash-4.3# ls -a
. .. .gitconfig .ssh git gogs-repositories
bash-4.3# ls -l .ssh
total 4
-rw------- 1 git git 23 Oct 1 22:39 environment
The Gogs server executes as the "git" user ID, and the home directory for that user is /data/git. As you recall we mounted $HOME/gogs as /data inside the container, and it's possible to inspect the file system from inside the container and outside the container (on your laptop) to see it's the same.
bash-4.3# su - git
Become the "git" user
2fa56ab52226:~$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/data/git/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /data/git/.ssh/id_rsa.
Your public key has been saved in /data/git/.ssh/id_rsa.pub.
Then we can run ssh-keygen to generate an SSH key. You can then add the generated id_rsa.pub to the authorized_keys file of other places, such as the SSH login to an account on a VPS that you're renting.
With the SSH key added to a server, you can then login without password:
2fa56ab52226:~$ ssh -l example-user example.org
The authenticity of host 'example.org (###.###.###.###)' can't be established.
RSA key fingerprint is SHA256:..........
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'example.org,###.###.###.###' (RSA) to the list of known hosts.
Welcome to Ubuntu ...
* Documentation: https://help.ubuntu.com/
Last login: Sat Oct 1 14:46:58 2016 from 108.213.68.139
Automatically pushing to a remote server
So far I've verified that the "git" user inside the Gogs Docker container can freely ssh over to the server. It's time to exploit this by setting up a git hook to push the repository to a remote.
The official way to mirror a git repository to another location is with this command:
$ git push --mirror git@example.com/new-location.git
Let's simply put that in the post-receive hook of the repository. Gogs makes this easy.
A "git hook" is a shell script that runs at certain moments of Git's actions. The post-receive hook is run after a commit has been received and has been fully processed.
You can go to this directory on your laptop:
$ ls ~/gogs/git/gogs-repositories/gogs-user/repository-name.git/hooks/
applypatch-msg.sample post-update.sample pre-push.sample update
commit-msg.sample pre-applypatch.sample pre-rebase.sample update.sample
post-receive pre-commit.sample prepare-commit-msg.sample
Or this directory inside the Docker container:
2fa56ab52226:~$ ls gogs-repositories/gogs-user/repository-name.git/hooks/
applypatch-msg.sample post-receive pre-applypatch.sample pre-push.sample prepare-commit-msg.sample update.sample
commit-msg.sample post-update.sample pre-commit.sample pre-rebase.sample update
And see that the files are the same.
In the Gogs web user interface, go to the home page of your project, then click on Settings, then click on Git Hooks. You'll see pre-receive and post-receive. Click on the pencil next to post-receive, and you'll get into a little text edit window.
For starters enter this:
#!/bin/bash
(
env
) >> $HOME/log.txt 2>&1
This prints the environment, sending it to a log file for your perusal. That log file will be visible both inside the Docker container, and outside, similarly to the commands above. Also as you edit the file you'll see the contents of post-receive change.
Make a change to your repository and check it in. Immediately this script should run and you'll be able to see output in log.txt.
Now change the post-receive script to this:
#!/bin/bash
git push --mirror ssh://remote-user@example.org/home/remote-user/git/repository-name-mirror.git
This gets right to the business of pushing the repository to your remote server. The URL used here is a correct SSH URL specifying the user name and host name for the remote repository.
If your git commit is run at the command line you'll see it execute as follows:
$ git push
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 311 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: To ssh://remote-user@visforvoltage.org/home/remote-user/git/repository-name-mirror.git
remote: 7eda04a..7297b40 master -> master
To http://localhost:10080/gogs-user/repository-name.git
7eda04a..7297b40 master -> master
Notice the two lines starting with "remote". That's the output of this post-receive hook, being copied back to your terminal.
Performing a build step on the server
My use case is to build the code on the server, then deploy that to another server. I won't go over the steps, but obviously we can put a post-receive hook script into the ``/home/remote-user/git/repository-name-mirror.git/hooks` directory. Make that script perform our build, and voila the job is done.
