Jekyll2020-01-14T07:33:14-06:00/feed.xmlJan’s GutterJan's GutterIntroducing clidev2019-10-13T05:01:00-05:002019-10-13T05:01:00-05:00/introducing-clidev<p>I’ve been using <a href="https://github.com/jangutter/clidev">clidev</a> to manage a set of
development environments on my macOS laptop. Currently it is a thin set of bash
scripting around <a href="https://www.docker.com">Docker</a>, but it can easily be adapted
to other container runtimes.</p>
<h2 id="origin-story">Origin Story</h2>
<p>My main development laptop has been running macOS for the last five years. It’s
the product of me being curious, and the fact that most of my work happens
remotely, and the tools are <em>mostly</em> compatible.</p>
<p>I started by hosting my local development environments that require Linux
inside VM’s. That worked until the sheer amount of different environments
became a pain to manage. Not to mention memory usage, disk footprint and
VirtualBox’s drivers that need constant updating.</p>
<p>At that point, I could have switched to <a href="https://www.vagrantup.com">Vagrant</a>,
but then I decided to try out Docker.</p>
<h2 id="what-docker-is-doing-behind-the-scenes">What Docker is doing behind the scenes</h2>
<p>Docker CE on macOS runs containers in a
<a href="https://github.com/linuxkit/linuxkit">linuxkit</a> VM inside a hypervisor called
<a href="https://github.com/moby/hyperkit">hyperkit</a>. If you want to see more technical
details, I recommend reading
<a href="http://collabnix.com/how-docker-for-mac-works-under-the-hood/">this blog post</a>.</p>
<p>You do drop quite a bit of flexibility by not running a fully virtualised OS,
but what you gain in management, latency and resource sharing is well worth it.</p>
<p>You only pay the cost of running the VM once, and Docker runs containers inside
a shared environment. The VM adds a bit of isolation between macOS and the
containers. That said, the many, many security implications of untrusted
containers still apply. Running untrusted containers on your laptop is just
<em>slightly</em> less hazardous that piping <code class="highlighter-rouge">curl</code>’s output into <code class="highlighter-rouge">sudo bash</code>.</p>
<p>This model neatly covers my use case.</p>
<h2 id="first-steps">First steps</h2>
<p>This is probably the simplest way to get an ephemeral container based on a
distribution:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">-it</span> <span class="nt">--rm</span> centos:centos7 /bin/bash <span class="nt">-l</span>
</code></pre></div></div>
<p>This command spawns a new container, runs bash as if it had been invoked as a
login and deletes the container on exit. However, there were a number of common
tasks that soon started to accumulate.</p>
<h2 id="dry">DRY</h2>
<p>I soon found out I needed scripts and files to organise the following tasks:</p>
<ul>
<li>Customise the images by adding new repos, packages and tweaking config.</li>
<li>Set up volume mounts, both common and specific to different environments.</li>
<li>Tweak command-line options slightly for different runtimes.</li>
</ul>
<p>I had more than 8 different single-line bash scripts just for running and
building the containers. At this point I really should have investigated
<a href="https://docs.docker.com/compose/">Docker Compose</a> but, I was not interested in
building a container-based application, I was more interested in collating
my runtime environments.</p>
<p>I wanted to apply a healthy dose of “Don’t Repeat Yourself” to the scripts, but
keep most of the flexibility. And more importantly, I’d like to replace docker
with <a href="https://podman.io">podman</a> when I eventually switch to a CentOS 8 desktop.</p>
<h2 id="clidev-quick-setup"><a href="https://github.com/jangutter/clidev">clidev</a> quick setup</h2>
<p>In your home directory (or where convenient), run the following:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/jangutter/clidev.git
git clone https://github.com/jangutter/clidev-env.git
<span class="nb">mkdir </span>dockermount <span class="c"># optional</span>
</code></pre></div></div>
<p>In the <code class="highlighter-rouge">clidev</code> directory, you could edit the <code class="highlighter-rouge">config.sh</code> script if you wish
to place the <code class="highlighter-rouge">clidev-env</code> directory somewhere else, or you wish to experiment
with other container runtimes.</p>
<p>At the minimum you should edit <code class="highlighter-rouge">clidev-conf/conf.d/default-config.sh</code> and
<code class="highlighter-rouge">clidev-conf/conf.d/default-mounts.sh</code> to set the default namespace, volume and
temporary filesystem mounts for the dev environments.</p>
<h2 id="basic-operation">Basic operation</h2>
<p><code class="highlighter-rouge">clidev</code>’s config is very much code. It’s bash scripts running other bash
scripts. <em>Do not import untrusted environments.</em> During normal operation,
<code class="highlighter-rouge">clidev</code> sources the common scripts in the <code class="highlighter-rouge">conf.d</code> directory, and scans through
all the directories in the <code class="highlighter-rouge">env</code> directory. Each subdirectory there corresponds
to an environment that can be built and run.</p>
<p><code class="highlighter-rouge">clidev.sh</code> is <em>not</em> meant to be put on a system path. Normal operation assumes
that you change your working directory to the <code class="highlighter-rouge">clidev</code> directory and run the
script from there.</p>
<h3 id="listing-available-environments">Listing available environments</h3>
<p>On my system, the available environments show up like this:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gp">[jangutter@laptop ~]$</span><span class="w"> </span><span class="nb">cd </span>clidev
<span class="gp">[jangutter@laptop clidev]$</span><span class="w"> </span>./clidev.sh list
<span class="go">Environment : Description
----------------------
bionic : Ubuntu 18.04 + Development Tools
centos7 : systemd enabled CentOS 7 + Development Tools
centos8 : CentOS 8 + Development Tools
f30 : Fedora 30 + Development Tools
f31 : Fedora 31 + Development Tools
jekyll : Jekyll build environment for jangutter.com
kolla-ansible : Kolla-ansible CLI
nova-queens-build : CentOS 7 + Nova Queens Dev
osc-queens : CentOS 7 + OpenStack Queens CLI
sf-3.1 : Software Factory 3.1 CLI
sf-3.2 : Software Factory 3.2 CLI
sf-3.3 : Software Factory 3.3 CLI
trusty : Ubuntu 14.04 + Development Tools
ubi7 : RHEL UBI 7
</span></code></pre></div></div>
<h3 id="building-the-image-for-an-environment">Building the image for an environment</h3>
<p>To build the docker image for an environment, use the <code class="highlighter-rouge">build</code> parameter:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gp">[jangutter@laptop clidev]$</span><span class="w"> </span>./clidev.sh build centos7
</code></pre></div></div>
<h3 id="running-the-environment">Running the environment</h3>
<p>And finally, the environment can be run with:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gp">[jangutter@laptop clidev]$</span><span class="w"> </span>./clidev.sh run centos7
</code></pre></div></div>
<p>If you want to peek under the hood, you can add <code class="highlighter-rouge">-d</code> as the first parameter to
enable tracing:</p>
<div class="language-console highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gp">[jangutter@laptop clidev]$</span><span class="w"> </span>./clidev.sh <span class="nt">-d</span> run centos7
</code></pre></div></div>
<p>Note that the <code class="highlighter-rouge">centos7</code> environment is a special systemd enabled container. To
exit it you should use the <code class="highlighter-rouge">poweroff</code> command, or stop it externally by using
<code class="highlighter-rouge">docker stop</code>.</p>
<h2 id="what-next">What next?</h2>
<p>I’ll gradually update <code class="highlighter-rouge">clidev</code> with better documentation and more examples. At
this moment it’s the fastest way for me to enter an environment to test
something specific to a distribution, or to run a specific build or install
environment.</p>I’ve been using clidev to manage a set of development environments on my macOS laptop. Currently it is a thin set of bash scripting around Docker, but it can easily be adapted to other container runtimes.First Post!2019-04-21T11:03:00-05:002019-04-21T11:03:00-05:00/first-post<p>I sacrificed one Easter weekend to finally teach myself
<a href="https://jekyllrb.com">Jekyll</a> and the requisite bits of HTML and CSS to come
up with a spartan enough site. The
<a href="/about">about page</a> contains the links to most
of the bits slapped together to generate this site.</p>
<p>I’m still a bit worried that I’m going to want to convert everything to ReST at
some point, but lets face it, if I procrastinate any longer I will truly be
eligible for the Internet’s Eternally Under Construction Award (Lifetime).</p>
<p>Thus far, it’s not been an entirely unpleasant experience (aside from npm
falling down like a massive Jenga tower) and I hope it doesn’t take more time
to maintain than to use.</p>
<p>Here we go!</p>I sacrificed one Easter weekend to finally teach myself Jekyll and the requisite bits of HTML and CSS to come up with a spartan enough site. The about page contains the links to most of the bits slapped together to generate this site.