content/articles/git-setup-own-server.md

   1 ---
   2 title: "Git Setup Your Own Server"
   3 date: 2021-06-13T12:07:49+02:00
   4 type: post
   5 author: "danix"
   6 excerpt: "I'll show you how to setup your own git server easily"
   7 featured_image: "uploads/2018/07/gitout.jpg"
   8 tags:
   9   - git
  10   - server
  11   - setup
  12   - howto
  13   - do it yourself
  14   - ssh
  15 categories:
  16   - code
  17   - diy
  18   - linux
  19 ---
  20
  21 Hello everyone,
  22
  23 I recently decided to move all my code under GIT, I've used it before and I've used also SVN, but I find GIT to be more straightforward in some aspects.
  24
  25 In order to use git I needed a place online where to store my projects, and I thought that github could be a good place, but the fact that you have to pay to keep a project private just didn't sound right in my opinion. Of course github is there to make money (specially now that M$ bought it), but I prefer to have a simpler setup and be able to do things my way as much as possible.
  26
  27 So I started planning what I wanted my git server to have. Here's a list:
  28
  29   * **Security** - I decided to make it work only under ssh, that way only someone who has the key can clone or access the repository. I also added an unprivileged git user that has only access to very few commands, so even if somebody manages to access through ssh he'll find himrself with only very few options available.
  30   * **Notifications** - my server already tells me a lot of what happens, so I wanted my git service to do the same. I implemented a mail service that notifies me every time a new repository is added or everytime there's a push to a repository.
  31   * **Automation** - I wanted to have less steps possible between creation of the project and deployment to production. Now in two steps I can create a repository and clone it to my local computer, and when I'm done I just need to push my modifications and the code is deployed automatically.
  32   * **Visibility** - I haven't yet decided if I want my code to be visible, so I haven't even started thinking about this possibility.
  33
  34 <!--more-->
  35
  36 ## Installation
  37
  38 Installing a git server is quite simple once you know how it works, on my server it was a matter of having a bare repository setup but in order to have the level of security that I wanted there were a few steps involved.
  39
  40 > A bit of a disclaimer here, I use a Slackware64-14.2 on my server and Slackware64-current on my laptop, so all the commands here worked for me but I can't be sure if the procedures that I followed will work on different distros with different setups. If you have any trouble following what I've done let me know in the comments and I'll try to help you.
  41
  42 I've added a new user and group to my server but before doing so I added /usr/bin/git-shell to /etc/shells in order to use it as login shell for my git user.
  43
  44 {{< highlight bash >}}
  45 echo "/usr/bin/git-shell" >> /etc/shells
  46 groupadd git
  47 mkdir /var/git
  48 useradd -d /var/git -g git -M -s /usr/bin/git-shell
  49 {{< /highlight >}}
  50
  51 now the user is all set and ready to be used. Next step will be to create the .ssh directory and the authorized_keys file to hold the keys for the developers that have to access the git server. Here's how I did it:
  52
  53 ```bash
  54 mkdir /var/git/.ssh
  55 touch /var/git/.ssh/authorized_keys
  56 chown -R git:git /var/git
  57 chmod 0700 /var/git/.ssh
  58 chmod 0600 /var/git/.ssh/authorized_keys
  59 ```
  60
  61 ok, now the files are in place and the permissions are correct for ssh to work well.
  62
  63 Let's head back to my working computer, I created an ssh keypair for my usual user and copied the public key to the authorized_keys file on the server. I won't go into much detail on how to do so, but just a suggestion, keep it without password, it'll be much faster to work later.
  64
  65 Since I have ssh access to the same server for my normal user I used the ~/.ssh/config file on my computer to set a new host that will ease my access routine for the git user as well as my regular user, that's my config (more or less):
  66
  67 ```bash
  68 cat ~/.ssh/config
  69 Host regular_ssh
  70         HostName server.tld
  71         User myuser
  72         IdentityFile ~/.ssh/id_rsa
  73
  74 Host git_ssh
  75         HostName server.tld
  76         User git
  77         IdentityFile ~/.ssh/git_rsa
  78 ```
  79
  80 Now when I need to access the server with my regular user I'll just run
  81
  82 `ssh regular_ssh`
  83
  84 and when I need to access as git user I'll run
  85
  86 `ssh git_ssh`
  87
  88 and ssh will take care of all the options and start the connection with the correct credentials for me. Neat!
  89
  90 Now that the access for the git user is setup we have one last thing to do before being able to use it. We'll give him only limited commands to use, That way the git user will be even more limited and much more secure. Inside the documentation shipped with git there's a lot of scripts to get you started with this, so we'll copy them inside a special directory called git-shell-commands, like this:
  91
  92 ```bash
  93 cp -R /usr/doc/git-2.14.4/contrib/git-shell-commands /var/git
  94 chown -R git:git /var/git
  95
  96 ```
  97
  98 Now we have 2 commands inside the git-shell-commands directory, list and help, the first will show all projects inside the /var/git directory and the other will show a simple help text and a list of all the commands available. Now to give you an example of how easy it is to add commands to the git-shell I will create a simple command that acts as the clear command, it will clean the screen, to do so, from inside the /var/git directory I did:
  99
 100 ```bash
 101 echo $(which clear) > git-shell-commands/clear
 102 chmod 0755 git-shell-commands/clear
 103
 104 ```
 105
 106 and now I have a "clear" command available for my git user. Another useful command will be "create" to add a repository and a "destroy" to remove it. Let's see them in the next page.
 107
 108 <!--nextpage-->
 109
 110 Let's focus on the two most important scripts for our git server, the **create** and the **delete** script:
 111
 112 ### create
 113
 114 {{< highlight bash "linenos=table" >}}
 115 #! /bin/bash
 116
 117 # usage:        create <PROJECT> - create a git bare repository named PROJECT.git
 118 #               this command will setup the repo and send a mail for confirmation.
 119
 120 GITDIR="/var/git"
 121 MULTIMAIL="/usr/doc/git-2.14.4/contrib/hooks/multimail/git_multimail.py"
 122 GITUSER="git"
 123 GITGRP="git"
 124
 125 function is_bare() {
 126         repodir=$1
 127         if "$(git --git-dir="$repodir" rev-parse --is-bare-repository)" = true
 128         then
 129                 true
 130         else
 131                 false
 132         fi
 133 }
 134
 135 function git_init() {
 136         PROJECT=$1
 137         echo "creating project \"${PROJECT}.git\""
 138         if [ ! -d ${GITDIR}/${PROJECT}.git ]; then
 139                 mkdir ${GITDIR}/${PROJECT}.git
 140         fi
 141         cd ${GITDIR}/${PROJECT}.git
 142         git init --bare
 143         mkdir custom-hooks
 144         ln -s $MULTIMAIL custom-hooks/
 145         touch hooks/post-receive
 146         cat > hooks/post-receive <<EOPR
 147 #!/bin/sh
 148 /usr/bin/pee ${GITDIR}/${PROJECT}.git/custom-hooks/deploy.sh \
 149         ${GITDIR}/${PROJECT}.git/custom-hooks/git_multimail.py
 150
 151 EOPR
 152        cat >> config <<EOT
 153
 154 [multimailhook]
 155         mailer = "sendmail"
 156         refchangeShowGraph = true
 157         mailingList = "receiver@someemail.tld"
 158         commitEmailFormat = "html"
 159         htmlInIntro = true
 160         htmlInFooter = true
 161         from = "sender@someemail.tld"
 162         administrator = "admin@someemail.tld"
 163         quiet = true
 164         logFile = "/var/log/multimail.log"
 165         errorLogFile = "/var/log/multimail_err.log"
 166 EOT
 167         touch custom-hooks/deploy.sh
 168         cat > custom-hooks/deploy.sh <<EODP
 169 #!/bin/bash
 170 # Directory where to deploy files from repository
 171 DPTARGET=""
 172 # Directory containing repository
 173 DPGIT_DIR="${GITDIR}/${PROJECT}.git"
 174 # Branch that is going to be deployed to server
 175 DPBRANCH="master"
 176
 177 while read oldrev newrev ref
 178 do
 179         # if DPTARGET is empty we don't want deploy for this project
 180         if [[ ! "" == \$DPTARGET ]]; then
 181                 # let's check that we are deploying to the correct branch
 182                 if [[ \$ref = refs/heads/\${DPBRANCH} ]]; then
 183                         echo "Ref \$ref received. Deploying \${DPBRANCH} branch to production..."
 184                         git --work-tree=\$DPTARGET --git-dir=\$DPGIT_DIR checkout -f $DPBRANCH
 185                         NOW=\$(date +"%d%m%Y-%H%M")
 186                         git tag release_\$NOW \$DPBRANCH
 187                         echo "   /==============================="
 188                         echo "   | DEPLOYMENT COMPLETED"
 189                         echo "   | Target branch: \$DPTARGET"
 190                         echo "   | Target folder: \$DPGIT_DIR"
 191                         echo "   | Tag name     : release_\$NOW"
 192                         echo "   \=============================="
 193                 else
 194                         echo "Ref \$ref received. Doing nothing: only the \${DPBRANCH} branch may be deployed on this server."
 195                 fi
 196         else
 197                 echo "Target directory not declared. Skipping deploy to server."
 198         fi
 199 done
 200
 201 EODP
 202         chmod 0755 hooks/post-receive custom-hooks/deploy.sh
 203         cd ${GITDIR}/
 204         chown -R ${GITUSER}:${GITGRP} ${GITDIR}/${PROJECT}.git
 205         echo "All done, you can now work on \"${PROJECT}.git\""
 206         exit 0
 207 }
 208
 209 if [ ! -z $1 ]; then
 210         PROJECT=$1
 211 else
 212         read -p 'Project name: ' PROJECT
 213 fi
 214
 215 if [ ! -d ${GITDIR}/${PROJECT}.git ]; then
 216         git_init $PROJECT
 217 else
 218         echo "Project directory ${PROJECT}.git already exists."
 219         if [ $(ls -A ${GITDIR}/${PROJECT}.git) ]; then
 220                 if is_bare ${GITDIR}/${PROJECT}.git
 221                 then
 222                         echo "looks like \"${PROJECT}.git\" is an existing git project directory, choose another name."
 223                         exit 171
 224                 else
 225                         echo "\"${PROJECT}.git\" is not empty, I can't create a Git Project in it. Choose another name."
 226                         exit 172
 227                 fi
 228         else
 229                 echo "\"${PROJECT}.git\" is an empty directory. Do you want to initialize a Git Project here? [y/N]"
 230                 read answer
 231                 case $answer in
 232                         Y|y)
 233                                 git_init $PROJECT
 234                                 ;;
 235                         N|n)
 236                                 echo "Aborting due to user request."
 237                                 exit 173
 238                                 ;;
 239                         *)
 240                                 # we assume no as default answer.
 241                                 echo "you said \"$answer\" which I don't understand, so to me is no. Aborting."
 242                                 exit 177
 243                                 ;;
 244                 esac
 245         fi
 246 fi
 247 {{< /highlight >}}
 248
 249 This is the create script, as you can see it's a bit complex because it will do a few things for me:
 250
 251   * create the bare repository using the argument provided on the command line or asking for a project name
 252   * check before creating the repo, if another repo with the same name exists or if a directory with files in it exists. That way I'll make sure to only create a repo inside an empty directory.
 253   * add a custom-hooks directory that will hold a link to the multimail.py script as well as my deploy.sh script
 254   * create a post-receive script that uses **pee** from the [moreutils][1] project to run multiple scripts at the same hook.
 255   * ensure sane permissions on the whole project directory.
 256   * always assume no (the safest option) when asking the user about the action to take.
 257
 258 So after saving this script as create inside the git-shell-commands directory, we'll give it executable permissions and we can move to the delete script.
 259
 260 {{< highlight bash >}}
 261 cat create-bare-repo.sh &gt; /var/git/git-shell-commands/create
 262 chown git:git -R /var/git/git-shell-commands
 263 chmod 0755 /var/git/git-shell-commands/create
 264 {{< /highlight >}}
 265
 266 ### delete
 267
 268 Let's see the delete script:
 269
 270 {{< highlight bash "linenos=table" >}}
 271 #! /bin/bash
 272
 273 # usage:        delete <REPOSITORY> - PERMANENTLY delete a repository if existing.
 274 #               CAREFUL, this action cannot be undone. This command will ask for confirmation.
 275
 276 GITDIR="/var/git"
 277
 278 function is_bare() {
 279         repodir=$1
 280         if "$(git --git-dir="$repodir" rev-parse --is-bare-repository)" = true
 281         then
 282                 true
 283         else
 284                 false
 285         fi
 286 }
 287
 288 if [ ! -z $1 ]; then
 289         PROJECT=$1
 290 else
 291         read -p 'Project to delete: ' PROJECT
 292 fi
 293
 294 if [ -d ${GITDIR}/${PROJECT}.git ]; then
 295         if [[ $(ls -A ${GITDIR}/${PROJECT}.git) ]]; then
 296                 if is_bare ${GITDIR}/${PROJECT}.git
 297                 then
 298                         echo "You are going to delete the git repository \"${PROJECT}.git\" Do you really want to continue? Note, this action cannot be reverted. [y/N]"
 299                         read delAnswer
 300                         case $delAnswer in
 301                                 Y|y)
 302                                         rm -rf ${PROJECT}.git
 303                                         ;;
 304                                 N|n)
 305                                         echo "Aborting due to user request."
 306                                         exit 173
 307                                         ;;
 308                                 *)
 309                                         echo "you said \"$delAnswer\" which I don't understand. Assuming No. Aborting."
 310                                         exit 177
 311                                         ;;
 312                         esac
 313                 else
 314                         echo "\"${PROJECT}.git\" doesn't look like a git repository. Check with your System Administrator."
 315                         exit 177
 316                 fi
 317         else
 318                 echo "\"${PROJECT}.git\" is an empty directory, Skipping. Check with your System Administrator."
 319                 exit 177
 320         fi
 321 fi
 322 {{< /highlight >}}
 323
 324 This script is much simpler than the previous one, it'll accept the name of the project as argument on the command line or will ask for it and will only delete it if it is a proper git repository, otherwise it will just exit with an error code.
 325
 326 I improved the way those scripts recognise a git repository from simply relying on the fact that there's a HEAD file inside the directory they're checking, which wasn't the best option, to using git itself to check if the directory is a bare repository. Much better!!
 327
 328 Since we are here let's modify the help command to make it show a short description of every available command.
 329
 330 {{< highlight bash "linenos=table" >}}
 331 #!/bin/sh
 332
 333 # usage:        help - Lists all the available commands
 334 #               help <command> - Detailled explanation of how "command" works
 335
 336 if tty -s
 337 then
 338         HELPTEXT="Hi $USER, Run 'help' for help, 'help <command>' for specific help on a command, run 'exit' to exit. Available commands:"
 339 else
 340         HELPTEXT="Hi $USER, Run 'help' for help, 'help <command>' for specific help on a command. Available commands:"
 341 fi
 342
 343 cd "$(dirname "$0")"
 344
 345 if [[ ! -z $1 ]]; then
 346         cmd=$1
 347         if [[ -f $cmd && -x $cmd ]]; then
 348                 awk 'NR>=3&&NR<=4' $cmd | cut -c 3-
 349         else
 350                 echo "command \"$cmd\" doesn't exists"
 351         fi
 352 else
 353         echo $HELPTEXT
 354         for cmd in *
 355         do
 356                 case "$cmd" in
 357                 help) ;;
 358                 *) [ -f "$cmd" ] && [ -x "$cmd" ] && echo "$cmd" ;;
 359                 esac
 360         done
 361 fi
 362 {{< /highlight >}}
 363
 364 The main thing I added is the support for a command line argument, now I'm able to run it by itself and display the usual output with a list of available commands, or followed by a command name to give a brief explanation like this:
 365
 366 {{< highlight bash >}}
 367 git> help
 368 Hi git, Run 'help' for help, 'help <command>' for specific help on a command, run 'exit' to exit. Available commands:
 369 clear
 370 create
 371 delete
 372 list
 373 git> help create
 374 usage:  create  - create a git bare repository named PROJECT.git
 375         this command will setup the repo and send a mail for confirmation.
 376 git>
 377 {{< /highlight >}}
 378
 379 Pretty nice isn't it?! Now it's much more user friendly, and to show the description I used awk and cut to parse the comment at the top of every script I have in the `git-shell-commands` directory.
 380
 381 In the next page we'll see the usual routine I follow when working with this new setup.
 382
 383 <!--nextpage-->
 384
 385 ## My GIT Routine
 386
 387 Let's say I had a new idea for a WordPress plugin, I can't wait to start writing, so the setup of the GIT environment should be as fast as possible. That's where my setup will come in handy. Let's open the terminal, I'll go inside my testing directory and from there I'll run:
 388
 389 {{< highlight bash >}}ssh git_ssh 'create awesomePlugin'
 390 creating project "awesomePlugin.git"
 391 Initialized empty Git repository in /var/git/awesomePlugin.git/
 392 All done, you can now work on "awesomePlugin.git"
 393 {{< /highlight >}}
 394
 395 The project is created, now I just need to clone it
 396
 397 {{< highlight bash >}}git clone ssh://git_ssh:/var/git/awesomePLugin.git
 398 Cloning into 'awesomePlugin'...
 399 warning: Looks like you cloned an empty repository.
 400 {{< /highlight >}}
 401
 402 And that's it, I now have a local and a remote copy of my git repository ready to work with.
 403
 404 Let's say I'm working on this plugin and I get to the point where I feel like I can use it on my own blog, I want the deploy to be as fast as the rest of the process was, so I'll ssh as my normal user and modify the `deploy.sh` script inside the `custom-hooks` directory for this project to add the working directory and the branch I want to use for deploy (you can use master, but it's better to use a different branch only for this pourpose, this way you can keep the production code stable and use master for experimenting until ready to deploy)
 405
 406 Inside the deploy.sh script I'll edit those 2 lines:
 407
 408 {{< highlight bash >}}# Directory where to deploy files from repository
 409 DPTARGET=""
 410 # Branch that is going to be deployed to server
 411 DPBRANCH="master"
 412 {{< /highlight >}}
 413
 414 adding `/var/www/wp-content/plugins/awesomePlugin` as `DPTARGET` and production as `DPBRANCH`.
 415
 416 now on my local system I'll add a new branch and use that before committing my stable code.
 417
 418 {{< highlight bash >}}git checkout -b production
 419 Switched to a new branch 'production'
 420
 421 git add .
 422
 423 git commit -m "awesomePlugin is even more awesome"
 424 [production (root-commit) a3885a4] awesomePlugin is even more awesome
 425  1 file changed, 1 insertion(+)
 426  create mode 100644 awesomePlugin.php
 427
 428 git push origin production
 429 Counting objects: 3, done.
 430 Writing objects: 100% (3/3), 240 bytes | 240.00 KiB/s, done.
 431 Total 3 (delta 0), reused 0 (delta 0)
 432 remote: Ref refs/heads/production received. Deploying production branch to production...
 433 remote:    /===============================
 434 remote:    | DEPLOYMENT COMPLETED
 435 remote:    | Target branch: /var/www/htdocs/wp-content/plugins/awesomePlugin
 436 remote:    | Target folder: /var/git/awesomePlugin.git
 437 remote:    | Tag name     : release_12072018-1110
 438 remote:    \==============================
 439 To ssh://git_ssh:/var/git/awesomePlugin.git
 440  * [new branch]      production -> production
 441 {{< /highlight >}}
 442
 443 And that's it, now my new plugin is ready to go live as soon as I activate it inside my WordPress admin area.
 444
 445 ## Conclusion
 446
 447 That's it for now, I'll add to this post as soon as I decide whether I want my code to be visible or not, but for now this is my setup and it's working greatly for me so far.
 448
 449 If you made it this far I hope you'll spend a couple more minutes to let me know what you think about this setup, if you use something similar or if you had any problems setting this up, I'll try and help you as much as I can of course.
 450
 451 [1]: https://joeyh.name/code/moreutils/