articles/git-setup-own-server.md

   1 ---
   2 title: "Git Setup Your Own Server"
   3 date: 2021-06-13T12:07:49+02:00
   4 type: post
   5 draft: false
   6 excerpt: "I'll show you how to setup your own git server easily"
   7 featured_image: "img/gitout.jpg"
   8 tags: ["git", "server", "setup", "howto", "do it yourself", "ssh"]
   9 categories: ["code", "diy", "linux"]
  10 author_name: "danix"
  11 ---
  12
  13 Hello everyone,
  14
  15 I recently decided to move all my code under GIT, I've used it before and I've used also SVN, but I find GIT to be more straightforward in some aspects.
  16
  17 In order to use git I needed a place online where to store my projects, and I thought that github could be a good place, but the fact that you have to pay to keep a project private just didn't sound right in my opinion. Of course github is there to make money (specially now that M$ bought it), but I prefer to have a simpler setup and be able to do things my way as much as possible.
  18
  19 So I started planning what I wanted my git server to have. Here's a list:
  20
  21   * **Security** - I decided to make it work only under ssh, that way only someone who has the key can clone or access the repository. I also added an unprivileged git user that has only access to very few commands, so even if somebody manages to access through ssh he'll find himrself with only very few options available.
  22   * **Notifications** - my server already tells me a lot of what happens, so I wanted my git service to do the same. I implemented a mail service that notifies me every time a new repository is added or everytime there's a push to a repository.
  23   * **Automation** - I wanted to have less steps possible between creation of the project and deployment to production. Now in two steps I can create a repository and clone it to my local computer, and when I'm done I just need to push my modifications and the code is deployed automatically.
  24   * **Visibility** - I haven't yet decided if I want my code to be visible, so I haven't even started thinking about this possibility.
  25
  26 <!--more-->
  27
  28 ## Installation
  29
  30 Installing a git server is quite simple once you know how it works, on my server it was a matter of having a bare repository setup but in order to have the level of security that I wanted there were a few steps involved.
  31
  32 > A bit of a disclaimer here, I use a Slackware64-14.2 on my server and Slackware64-current on my laptop, so all the commands here worked for me but I can't be sure if the procedures that I followed will work on different distros with different setups. If you have any trouble following what I've done let me know in the comments and I'll try to help you.
  33
  34 I've added a new user and group to my server but before doing so I added /usr/bin/git-shell to /etc/shells in order to use it as login shell for my git user.
  35
  36 ```
  37 echo "/usr/bin/git-shell" >> /etc/shells
  38 groupadd git
  39 mkdir /var/git
  40 useradd -d /var/git -g git -M -s /usr/bin/git-shell
  41 ```
  42
  43 now the user is all set and ready to be used. Next step will be to create the .ssh directory and the authorized_keys file to hold the keys for the developers that have to access the git server. Here's how I did it:
  44
  45 ```
  46 mkdir /var/git/.ssh
  47 touch /var/git/.ssh/authorized_keys
  48 chown -R git:git /var/git
  49 chmod 0700 /var/git/.ssh
  50 chmod 0600 /var/git/.ssh/authorized_keys
  51 ```
  52
  53 ok, now the files are in place and the permissions are correct for ssh to work well.
  54
  55 Let's head back to my working computer, I created an ssh keypair for my usual user and copied the public key to the authorized_keys file on the server. I won't go into much detail on how to do so, but just a suggestion, keep it without password, it'll be much faster to work later.
  56
  57 Since I have ssh access to the same server for my normal user I used the ~/.ssh/config file on my computer to set a new host that will ease my access routine for the git user as well as my regular user, that's my config (more or less):
  58
  59 ```
  60 cat ~/.ssh/config
  61 Host regular_ssh
  62         HostName server.tld
  63         User myuser
  64         IdentityFile ~/.ssh/id_rsa
  65
  66 Host git_ssh
  67         HostName server.tld
  68         User git
  69         IdentityFile ~/.ssh/git_rsa
  70 ```
  71
  72 Now when I need to access the server with my regular user I'll just run
  73
  74 `ssh regular_ssh`
  75
  76 and when I need to access as git user I'll run
  77
  78 `ssh git_ssh`
  79
  80 and ssh will take care of all the options and start the connection with the correct credentials for me. Neat!
  81
  82 Now that the access for the git user is setup we have one last thing to do before being able to use it. We'll give him only limited commands to use, That way the git user will be even more limited and much more secure. Inside the documentation shipped with git there's a lot of scripts to get you started with this, so we'll copy them inside a special directory called git-shell-commands, like this:
  83
  84 ```
  85 cp -R /usr/doc/git-2.14.4/contrib/git-shell-commands /var/git
  86 chown -R git:git /var/git
  87
  88 ```
  89
  90 Now we have 2 commands inside the git-shell-commands directory, list and help, the first will show all projects inside the /var/git directory and the other will show a simple help text and a list of all the commands available. Now to give you an example of how easy it is to add commands to the git-shell I will create a simple command that acts as the clear command, it will clean the screen, to do so, from inside the /var/git directory I did:
  91
  92 ```
  93 echo $(which clear) > git-shell-commands/clear
  94 chmod 0755 git-shell-commands/clear
  95
  96 ```
  97
  98 and now I have a "clear" command available for my git user. Another useful command will be "create" to add a repository and a "destroy" to remove it. Let's see them in the next page.
  99
 100 <!--nextpage-->
 101
 102 Let's focus on the two most important scripts for our git server, the **create** and the **delete** script:
 103
 104 ### create
 105
 106 <pre class="wp-block-code language-bash"><code>#! /bin/bash
 107
 108 # usage:        create <PROJECT> - create a git bare repository named PROJECT.git
 109 #               this command will setup the repo and send a mail for confirmation.
 110
 111 GITDIR="/var/git"
 112 MULTIMAIL="/usr/doc/git-2.14.4/contrib/hooks/multimail/git_multimail.py"
 113 GITUSER="git"
 114 GITGRP="git"
 115
 116 function is_bare() {
 117         repodir=$1
 118         if "$(git --git-dir="$repodir" rev-parse --is-bare-repository)" = true
 119         then
 120                 true
 121         else
 122                 false
 123         fi
 124 }
 125
 126 function git_init() {
 127         PROJECT=$1
 128         echo "creating project \"${PROJECT}.git\""
 129         if [ ! -d ${GITDIR}/${PROJECT}.git ]; then
 130                 mkdir ${GITDIR}/${PROJECT}.git
 131         fi
 132         cd ${GITDIR}/${PROJECT}.git
 133         git init --bare
 134         mkdir custom-hooks
 135         ln -s $MULTIMAIL custom-hooks/
 136         touch hooks/post-receive
 137         cat > hooks/post-receive <<EOPR
 138 #!/bin/sh
 139 /usr/bin/pee ${GITDIR}/${PROJECT}.git/custom-hooks/deploy.sh \
 140         ${GITDIR}/${PROJECT}.git/custom-hooks/git_multimail.py
 141
 142 EOPR
 143        cat >> config <<EOT
 144
 145 [multimailhook]
 146         mailer = "sendmail"
 147         refchangeShowGraph = true
 148         mailingList = "receiver@someemail.tld"
 149         commitEmailFormat = "html"
 150         htmlInIntro = true
 151         htmlInFooter = true
 152         from = "sender@someemail.tld"
 153         administrator = "admin@someemail.tld"
 154         quiet = true
 155         logFile = "/var/log/multimail.log"
 156         errorLogFile = "/var/log/multimail_err.log"
 157 EOT
 158         touch custom-hooks/deploy.sh
 159         cat > custom-hooks/deploy.sh <<EODP
 160 #!/bin/bash
 161 # Directory where to deploy files from repository
 162 DPTARGET=""
 163 # Directory containing repository
 164 DPGIT_DIR="${GITDIR}/${PROJECT}.git"
 165 # Branch that is going to be deployed to server
 166 DPBRANCH="master"
 167
 168 while read oldrev newrev ref
 169 do
 170         # if DPTARGET is empty we don't want deploy for this project
 171         if [[ ! "" == \$DPTARGET ]]; then
 172                 # let's check that we are deploying to the correct branch
 173                 if [[ \$ref = refs/heads/\${DPBRANCH} ]]; then
 174                         echo "Ref \$ref received. Deploying \${DPBRANCH} branch to production..."
 175                         git --work-tree=\$DPTARGET --git-dir=\$DPGIT_DIR checkout -f $DPBRANCH
 176                         NOW=\$(date +"%d%m%Y-%H%M")
 177                         git tag release_\$NOW \$DPBRANCH
 178                         echo "   /==============================="
 179                         echo "   | DEPLOYMENT COMPLETED"
 180                         echo "   | Target branch: \$DPTARGET"
 181                         echo "   | Target folder: \$DPGIT_DIR"
 182                         echo "   | Tag name     : release_\$NOW"
 183                         echo "   \=============================="
 184                 else
 185                         echo "Ref \$ref received. Doing nothing: only the \${DPBRANCH} branch may be deployed on this server."
 186                 fi
 187         else
 188                 echo "Target directory not declared. Skipping deploy to server."
 189         fi
 190 done
 191
 192 EODP
 193         chmod 0755 hooks/post-receive custom-hooks/deploy.sh
 194         cd ${GITDIR}/
 195         chown -R ${GITUSER}:${GITGRP} ${GITDIR}/${PROJECT}.git
 196         echo "All done, you can now work on \"${PROJECT}.git\""
 197         exit 0
 198 }
 199
 200 if [ ! -z $1 ]; then
 201         PROJECT=$1
 202 else
 203         read -p 'Project name: ' PROJECT
 204 fi
 205
 206 if [ ! -d ${GITDIR}/${PROJECT}.git ]; then
 207         git_init $PROJECT
 208 else
 209         echo "Project directory ${PROJECT}.git already exists."
 210         if [ $(ls -A ${GITDIR}/${PROJECT}.git) ]; then
 211                 if is_bare ${GITDIR}/${PROJECT}.git
 212                 then
 213                         echo "looks like \"${PROJECT}.git\" is an existing git project directory, choose another name."
 214                         exit 171
 215                 else
 216                         echo "\"${PROJECT}.git\" is not empty, I can't create a Git Project in it. Choose another name."
 217                         exit 172
 218                 fi
 219         else
 220                 echo "\"${PROJECT}.git\" is an empty directory. Do you want to initialize a Git Project here? [y/N]"
 221                 read answer
 222                 case $answer in
 223                         Y|y)
 224                                 git_init $PROJECT
 225                                 ;;
 226                         N|n)
 227                                 echo "Aborting due to user request."
 228                                 exit 173
 229                                 ;;
 230                         *)
 231                                 # we assume no as default answer.
 232                                 echo "you said \"$answer\" which I don't understand, so to me is no. Aborting."
 233                                 exit 177
 234                                 ;;
 235                 esac
 236         fi
 237 fi</code></pre>
 238
 239 This is the create script, as you can see it's a bit complex because it will do a few things for me:
 240
 241   * create the bare repository using the argument provided on the command line or asking for a project name
 242   * check before creating the repo, if another repo with the same name exists or if a directory with files in it exists. That way I'll make sure to only create a repo inside an empty directory.
 243   * add a custom-hooks directory that will hold a link to the multimail.py script as well as my deploy.sh script
 244   * create a post-receive script that uses **pee** from the [moreutils][1] project to run multiple scripts at the same hook.
 245   * ensure sane permissions on the whole project directory.
 246   * always assume no (the safest option) when asking the user about the action to take.
 247
 248 So after saving this script as create inside the git-shell-commands directory, we'll give it executable permissions and we can move to the delete script.
 249
 250 <pre class="wp-block-code language-bash"><code># cat create-bare-repo.sh &gt; /var/git/git-shell-commands/create
 251 # chown git:git -R /var/git/git-shell-commands
 252 # chmod 0755 /var/git/git-shell-commands/create</code></pre>
 253
 254 ### delete
 255
 256 Let's see the delete script:
 257
 258 <pre class="wp-block-code language-bash"><code>#! /bin/bash
 259
 260 # usage:        delete <REPOSITORY> - PERMANENTLY delete a repository if existing.
 261 #               CAREFUL, this action cannot be undone. This command will ask for confirmation.
 262
 263 GITDIR="/var/git"
 264
 265 function is_bare() {
 266         repodir=$1
 267         if "$(git --git-dir="$repodir" rev-parse --is-bare-repository)" = true
 268         then
 269                 true
 270         else
 271                 false
 272         fi
 273 }
 274
 275 if [ ! -z $1 ]; then
 276         PROJECT=$1
 277 else
 278         read -p 'Project to delete: ' PROJECT
 279 fi
 280
 281 if [ -d ${GITDIR}/${PROJECT}.git ]; then
 282         if [[ $(ls -A ${GITDIR}/${PROJECT}.git) ]]; then
 283                 if is_bare ${GITDIR}/${PROJECT}.git
 284                 then
 285                         echo "You are going to delete the git repository \"${PROJECT}.git\" Do you really want to continue? Note, this action cannot be reverted. [y/N]"
 286                         read delAnswer
 287                         case $delAnswer in
 288                                 Y|y)
 289                                         rm -rf ${PROJECT}.git
 290                                         ;;
 291                                 N|n)
 292                                         echo "Aborting due to user request."
 293                                         exit 173
 294                                         ;;
 295                                 *)
 296                                         echo "you said \"$delAnswer\" which I don't understand. Assuming No. Aborting."
 297                                         exit 177
 298                                         ;;
 299                         esac
 300                 else
 301                         echo "\"${PROJECT}.git\" doesn't look like a git repository. Check with your System Administrator."
 302                         exit 177
 303                 fi
 304         else
 305                 echo "\"${PROJECT}.git\" is an empty directory, Skipping. Check with your System Administrator."
 306                 exit 177
 307         fi
 308 fi</code></pre>
 309
 310 This script is much simpler than the previous one, it'll accept the name of the project as argument on the command line or will ask for it and will only delete it if it is a proper git repository, otherwise it will just exit with an error code.
 311
 312 I improved the way those scripts recognise a git repository from simply relying on the fact that there's a HEAD file inside the directory they're checking, which wasn't the best option, to using git itself to check if the directory is a bare repository. Much better!!
 313
 314 Since we are here let's modify the help command to make it show a short description of every available command.
 315
 316 <pre class="wp-block-code language-bash"><code>#!/bin/sh
 317
 318 # usage:        help - Lists all the available commands
 319 #               help <command> - Detailled explanation of how "command" works
 320
 321 if tty -s
 322 then
 323         HELPTEXT="Hi $USER, Run 'help' for help, 'help <command>' for specific help on a command, run 'exit' to exit. Available commands:"
 324 else
 325         HELPTEXT="Hi $USER, Run 'help' for help, 'help <command>' for specific help on a command. Available commands:"
 326 fi
 327
 328 cd "$(dirname "$0")"
 329
 330 if [[ ! -z $1 ]]; then
 331         cmd=$1
 332         if [[ -f $cmd && -x $cmd ]]; then
 333                 awk 'NR>=3&&NR<=4' $cmd | cut -c 3-
 334         else
 335                 echo "command \"$cmd\" doesn't exists"
 336         fi
 337 else
 338         echo $HELPTEXT
 339         for cmd in *
 340         do
 341                 case "$cmd" in
 342                 help) ;;
 343                 *) [ -f "$cmd" ] && [ -x "$cmd" ] && echo "$cmd" ;;
 344                 esac
 345         done
 346 fi</code></pre>
 347
 348 The main thing I added is the support for a command line argument, now I'm able to run it by itself and display the usual output with a list of available commands, or followed by a command name to give a brief explanation like this:
 349
 350 <pre class="wp-block-code language-bash"><code>git> help
 351 Hi git, Run 'help' for help, 'help <command>' for specific help on a command, run 'exit' to exit. Available commands:
 352 clear
 353 create
 354 delete
 355 list
 356 git> help create
 357 usage:  create  - create a git bare repository named PROJECT.git
 358         this command will setup the repo and send a mail for confirmation.
 359 git>
 360 </code></pre>
 361
 362 Pretty nice isn't it?! Now it's much more user friendly, and to show the description I used awk and cut to parse the comment at the top of every script I have in the `git-shell-commands` directory.
 363
 364 In the next page we'll see the usual routine I follow when working with this new setup.
 365
 366 <!--nextpage-->
 367
 368 ## My GIT Routine
 369
 370 Let's say I had a new idea for a WordPress plugin, I can't wait to start writing, so the setup of the GIT environment should be as fast as possible. That's where my setup will come in handy. Let's open the terminal, I'll go inside my testing directory and from there I'll run:
 371
 372 <pre class="wp-block-code language-bash"><code>ssh git_ssh 'create awesomePlugin'
 373 creating project "awesomePlugin.git"
 374 Initialized empty Git repository in /var/git/awesomePlugin.git/
 375 All done, you can now work on "awesomePlugin.git"
 376 </code></pre>
 377
 378 The project is created, now I just need to clone it
 379
 380 <pre class="wp-block-code language-bash"><code>git clone ssh://git_ssh:/var/git/awesomePLugin.git
 381 Cloning into 'awesomePlugin'...
 382 warning: Looks like you cloned an empty repository.
 383 </code></pre>
 384
 385 And that's it, I now have a local and a remote copy of my git repository ready to work with.
 386
 387 Let's say I'm working on this plugin and I get to the point where I feel like I can use it on my own blog, I want the deploy to be as fast as the rest of the process was, so I'll ssh as my normal user and modify the `deploy.sh` script inside the `custom-hooks` directory for this project to add the working directory and the branch I want to use for deploy (you can use master, but it's better to use a different branch only for this pourpose, this way you can keep the production code stable and use master for experimenting until ready to deploy)
 388
 389 Inside the deploy.sh script I'll edit those 2 lines:
 390
 391 <pre class="wp-block-code language-bash"><code># Directory where to deploy files from repository
 392 DPTARGET=""
 393 # Branch that is going to be deployed to server
 394 DPBRANCH="master"
 395 </code></pre>
 396
 397 adding `/var/www/wp-content/plugins/awesomePlugin` as `DPTARGET` and production as `DPBRANCH`.
 398
 399 now on my local system I'll add a new branch and use that before committing my stable code.
 400
 401 <pre class="wp-block-code language-bash"><code>git checkout -b production
 402 Switched to a new branch 'production'
 403
 404 git add .
 405
 406 git commit -m "awesomePlugin is even more awesome"
 407 [production (root-commit) a3885a4] awesomePlugin is even more awesome
 408  1 file changed, 1 insertion(+)
 409  create mode 100644 awesomePlugin.php
 410
 411 git push origin production
 412 Counting objects: 3, done.
 413 Writing objects: 100% (3/3), 240 bytes | 240.00 KiB/s, done.
 414 Total 3 (delta 0), reused 0 (delta 0)
 415 remote: Ref refs/heads/production received. Deploying production branch to production...
 416 remote:    /===============================
 417 remote:    | DEPLOYMENT COMPLETED
 418 remote:    | Target branch: /var/www/htdocs/wp-content/plugins/awesomePlugin
 419 remote:    | Target folder: /var/git/awesomePlugin.git
 420 remote:    | Tag name     : release_12072018-1110
 421 remote:    \==============================
 422 To ssh://git_ssh:/var/git/awesomePlugin.git
 423  * [new branch]      production -> production
 424 </code></pre>
 425
 426 And that's it, now my new plugin is ready to go live as soon as I activate it inside my WordPress admin area.
 427
 428 ## Conclusion
 429
 430 That's it for now, I'll add to this post as soon as I decide whether I want my code to be visible or not, but for now this is my setup and it's working greatly for me so far.
 431
 432 If you made it this far I hope you'll spend a couple more minutes to let me know what you think about this setup, if you use something similar or if you had any problems setting this up, I'll try and help you as much as I can of course.
 433
 434  [1]: https://joeyh.name/code/moreutils/