Web dev in DC http://ross.karchner.com
612 stories
·
15 followers

Ars Technica takes another trip to the past

1 Share

After Andrew Cunningham’s experiment in 2014 with a PowerBook G4 running Mac OS 9.2.2, another tech writer from Ars Technica goes vintage, with an even older, but more fascinating setup: a Macintosh IIsi (introduced in late 1990), running System 7.5.5, and connected to a Macintosh Portrait Display (similar vintage). Back then, I wasn’t particularly satisfied with how Cunningham approached his exploration, and I wrote an article in response detailing my observations: Actual work on vintage Macs is possible.

This time I must say I enjoyed Chris Wilkinson’s article so much more than I did Cunningham’s. Chris’ approach seemed more open, and he sounded definitely more patient and willing to deal with the most challenging aspects of using a 28-year-old machine today. His is an excellent write-up of the experience, and I urge you to give it a read. As for my personal observations, I have very little to add.

In his conclusion, Chris writes [emphasis mine]:

In contrast, taking the IIsi through its paces was a joy. The limitations of the machine, with barely enough power to run more than one application at once, demands your attention to be 100 percent devoted to any single task. Paradoxically, it often felt like I was more productive with significantly fewer resources at hand. It captured and holds my attention on a single problem, rather than splitting my attention across dozens of unrelated tasks. Coming in with low expectations and knowing roughly what 20MHz can do for me these days, I came away from my sojourn pleasantly surprised.

This is something I have experienced myself numerous times when using my vintage Macs, and it’s the main reason I generally prefer to bring a vintage Mac with me when I’m not working from home (if you follow me on Twitter, you may have noticed the occasional ‘Today’s vintage mobile office‘ photo). It really helps me stay focused, especially when I need to do some creative writing.

As I said, I really liked how Chris approached his vintage challenge. A couple of things I may have done differently: first, I’d have probably got more performance out of the IIsi by keeping it on System 7.1 — less feature-rich than 7.5.5, but also less RAM-hungry. And the second thing is related to music. Instead of pushing the Macintosh IIsi to its limits by handling MP3 files, I would have looked for an external SCSI CD-ROM drive, and just listened to audio CDs while working (the Control Strip had a handy module for quick access to CD playback controls). But this is just nitpicking.

Enjoy the article: 1990, meet 2018: How far does 20MHz of Macintosh IIsi power go today? by Chris Wilkinson.

 

 



Read the whole story
rosskarchner
8 days ago
reply
DC-ish
Share this story
Delete

Terraform becomes a programming language

1 Share

Declarative languages failure

Approach that in my eyes failed again and again is to start with your own declarative language and then with time grow the language. (SQL being among notable exceptions)

Puppet is the best example. map and each, added in Puppet 4.0.0 are, in my opinion, just two in a sea of evidence that the envisioned simple format have failed to handle the needs of the real world.

Ansible’s loop looks bad as the whole idea of making top levels of programs in YAML based syntax (and the rest in Python).

In my opinion, it makes more sense to create a language first and then libraries for it, not a library and then a language around it.

My hope for Terraform

I think Terraform guys are smart. Among other things, it manifests in implementing data sources. Data sources make Terraform much more flexible. I think it’s very clever.

Terraform, which started declarative, are now inventing their own programming language. They are going the way of Puppet and Ansible. I hope they can do better, in this awkward situation: there are quite a lot of constraints on the programming language because of the existing syntax and semantics.

Happy coding, everyone!

 



Read the whole story
rosskarchner
15 days ago
reply
DC-ish
Share this story
Delete

Walkthrough for Portable Services

1 Share

Portable Services with systemd v239

systemd v239 contains a great number of new features. One of them is first class support for Portable Services. In this blog story I'd like to shed some light on what they are and why they might be interesting for your application.

What are "Portable Services"?

The "Portable Service" concept takes inspiration from classic chroot() environments as well as container management and brings a number of their features to more regular system service management.

While the definition of what a "container" really is is hotly debated, I figure people can generally agree that the "container" concept primarily provides two major features:

  1. Resource bundling: a container generally brings its own file system tree along, bundling any shared libraries and other resources it might need along with the main service executables.

  2. Isolation and sand-boxing: a container operates in a name-spaced environment that is relatively detached from the host. Besides living in its own file system namespace it usually also has its own user database, process tree and so on. Access from the container to the host is limited with various security technologies.

Of these two concepts the first one is also what traditional UNIX chroot() environments are about.

Both resource bundling and isolation/sand-boxing are concepts systemd has implemented to varying degrees for a longer time. Specifically, RootDirectory= and RootImage= have been around for a long time, and so have been the various sand-boxing features systemd provides. The Portable Services concept builds on that, putting these features together in a new, integrated way to make them more accessible and usable.

OK, so what precisely is a "Portable Service"?

Much like a container image, a portable service on disk can be just a directory tree that contains service executables and all their dependencies, in a hierarchy resembling the normal Linux directory hierarchy. A portable service can also be a raw disk image, containing a file system containing such a tree (which can be mounted via a loop-back block device), or multiple file systems (in which case they need to follow the Discoverable Partitions Specification and be located within a GPT partition table). Regardless whether the portable service on disk is a simple directory tree or a raw disk image, let's call this concept the portable service image.

Such images can be generated with any tool typically used for the purpose of installing OSes inside some directory, for example dnf --installroot= or debootstrap. There are very few requirements made on these trees, except the following two:

  1. The tree should carry systemd unit files for relevant services in them.

  2. The tree should carry /usr/lib/os-release (or /etc/os-release) OS release information.

Of course, as you might notice, OS trees generated from any of today's big distributions generally qualify for these two requirements without any further modification, as pretty much all of them adopted /usr/lib/os-release and tend to ship their major services with systemd unit files.

A portable service image generated like this can be "attached" or "detached" from a host:

  1. "Attaching" an image to a host is done through the new portablectl attach command. This command dissects the image, reading the os-release information, and searching for unit files in them. It then copies relevant unit files out of the images and into /etc/systemd/system/. After that it augments any copied service unit files in two ways: a drop-in adding a RootDirectory= or RootImage= line is added in so that even though the unit files are now available on the host when started they run the referenced binaries from the image. It also symlinks in a second drop-in which is called a "profile", which is supposed to carry additional security settings to enforce on the attached services, to ensure the right amount of sand-boxing.

  2. "Detaching" an image from the host is done through portable detach. It reverses the steps above: the unit files copied out are removed again, and so are the two drop-in files generated for them.

While a portable service is attached its relevant unit files are made available on the host like any others: they will appear in systemctl list-unit-files, you can enable and disable them, you can start them and stop them. You can extend them with systemctl edit. You can introspect them. You can apply resource management to them like to any other service, and you can process their logs like any other service and so on. That's because they really are native systemd services, except that they have 'twist' if you so will: they have tougher security by default and store their resources in a root directory or image.

And that's already the essence of what Portable Services are.

A couple of interesting points:

  1. Even though the focus is on shipping service unit files in portable service images, you can actually ship timer units, socket units, target units, path units in portable services too. This means you can very naturally do time, socket and path based activation. It's also entirely fine to ship multiple service units in the same image, in case you have more complex applications.

  2. This concept introduces zero new metadata. Unit files are an existing concept, as are os-release files, and — in case you opt for raw disk images — GPT partition tables are already established too. This also means existing tools to generate images can be reused for building portable service images to a large degree as no completely new artifact types need to be generated.

  3. Because the Portable Service concepts introduces zero new metadata and just builds on existing security and resource bundling features of systemd it's implemented in a set of distinct tools, relatively disconnected from the rest of systemd. Specifically, the main user-facing command is portablectl, and the actual operations are implemented in systemd-portabled.service. If you so will, portable services are a true add-on to systemd, just making a specific work-flow nicer to use than with the basic operations systemd otherwise provides. Also note that systemd-portabled provides bus APIs accessible to any program that wants to interface with it, portablectl is just one tool that happens to be shipped along with systemd.

  4. Since Portable Services are a feature we only added very recently we wanted to keep some freedom to make changes still. Due to that we decided to install the portablectl command into /usr/lib/systemd/ for now, so that it does not appear in $PATH by default. This means, for now you have to invoke it with a full path: /usr/lib/systemd/portablectl. We expect to move it into /usr/bin/ very soon though, and make it a fully supported interface of systemd.

  5. You may wonder which unit files contained in a portable service image are the ones considered "relevant" and are actually copied out by the portablectl attach operation. Currently, this is derived from the image name. Let's say you have an image stored in a directory /var/lib/portables/foobar_4711/ (or alternatively in a raw image /var/lib/portables/foobar_4711.raw). In that case the unit files copied out match the pattern foobar*.service, foobar*.socket, foobar*.target, foobar*.path, foobar*.timer.

  6. The Portable Services concept does not define any specific method how images get on the deployment machines, that's entirely up to administrators. You can just scp them there, or wget them. You could even package them as RPMs and then deploy them with dnf if you feel adventurous.

  7. Portable service images can reside in any directory you like. However, if you place them in /var/lib/portables/ then portablectl will find them easily and can show you a list of images you can attach and suchlike.

  8. Attaching a portable service image can be done persistently, so that it remains attached on subsequent boots (which is the default), or it can be attached only until the next reboot, by passing --runtime to portablectl.

  9. Because portable service images are ultimately just regular OS images, it's natural and easy to build a single image that can be used in three different ways:

    1. It can be attached to any host as a portable service image.

    2. It can be booted as OS container, for example in a container manager like systemd-nspawn.

    3. It can be booted as host system, for example on bare metal or in a VM manager.

    Of course, to qualify for the latter two the image needs to contain more than just the service binaries, the os-release file and the unit files. To be bootable an OS container manager such as systemd-nspawn the image needs to contain an init system of some form, for example systemd. To be bootable on bare metal or as VM it also needs a boot loader of some form, for example systemd-boot.

Profiles

In the previous section the "profile" concept was briefly mentioned. Since they are a major feature of the Portable Services concept, they deserve some focus. A "profile" is ultimately just a pre-defined drop-in file for unit files that are attached to a host. They are supposed to mostly contain sand-boxing and security settings, but may actually contain any other settings, too. When a portable service is attached a suitable profile has to be selected. If none is selected explicitly, the default profile called default is used. systemd ships with four different profiles out of the box:

  1. The default profile provides a medium level of security. It contains settings to drop capabilities, enforce system call filters, restrict many kernel interfaces and mount various file systems read-only.

  2. The strict profile is similar to the default profile, but generally uses the most restrictive sand-boxing settings. For example networking is turned off and access to AF_NETLINK sockets is prohibited.

  3. The trusted profile is the least strict of them all. In fact it makes almost no restrictions at all. A service run with this profile has basically full access to the host system.

  4. The nonetwork profile is mostly identical to default, but also turns off network access.

Note that the profile is selected at the time the portable service image is attached, and it applies to all service files attached, in case multiple are shipped in the same image. Thus, the sand-boxing restriction to enforce are selected by the administrator attaching the image and not the image vendor.

Additional profiles can be defined easily by the administrator, if needed. We might also add additional profiles sooner or later to be shipped with systemd out of the box.

What's the use-case for this? If I have containers, why should I bother?

Portable Services are primarily intended to cover use-cases where code should more feel like "extensions" to the host system rather than live in disconnected, separate worlds. The profile concept is supposed to be tunable to the exact right amount of integration or isolation needed for an application.

In the container world the concept of "super-privileged containers" has been touted a lot, i.e. containers that run with full privileges. It's precisely that use-case that portable services are intended for: extensions to the host OS, that default to isolation, but can optionally get as much access to the host as needed, and can naturally take benefit of the full functionality of the host. The concept should hence be useful for all kinds of low-level system software that isn't shipped with the OS itself but needs varying degrees of integration with it. Besides servers and appliances this should be particularly interesting for IoT and embedded devices.

Because portable services are just a relatively small extension to the way system services are otherwise managed, they can be treated like regular service for almost all use-cases: they will appear along regular services in all tools that can introspect systemd unit data, and can be managed the same way when it comes to logging, resource management, runtime life-cycles and so on.

Portable services are a very generic concept. While the original use-case is OS extensions, it's of course entirely up to you and other users to use them in a suitable way of your choice.

Walkthrough

Let's have a look how this all can be used. We'll start with building a portable service image from scratch, before we attach, enable and start it on a host.

Building a Portable Service image

As mentioned, you can use any tool you like that can create OS trees or raw images for building Portable Service images, for example debootstrap or dnf --installroot=. For this example walkthrough run we'll use mkosi, which is ultimately just a fancy wrapper around dnf and debootstrap but makes a number of things particularly easy when repetitively building images from source trees.

I have pushed everything necessary to reproduce this walkthrough locally to a GitHub repository. Let's check it out:

$ git clone https://github.com/systemd/portable-walkthrough.git

Let's have a look in the repository:

  1. First of all, walkthroughd.c is the main source file of our little service. To keep things simple it's written in C, but it could be in any language of your choice. The daemon as implemented won't do much: it just starts up and waits for SIGTERM, at which point it will shut down. It's ultimately useless, but hopefully illustrates how this all fits together. The C code has no dependencies besides libc.

  2. walkthroughd.service is a systemd unit file that starts our little daemon. It's a simple service, hence the unit file is trivial.

  3. Makefile is a short make build script to build the daemon binary. It's pretty trivial, too: it just takes the C file and builds a binary from it. It can also install the daemon. It places the binary in /usr/local/lib/walkthroughd/walkthroughd (why not in /usr/local/bin? because it's not a user-facing binary but a system service binary), and its unit file in /usr/local/lib/systemd/walkthroughd.service. If you want to test the daemon on the host we can now simply run make and then ./walkthroughd in order to check everything works.

  4. mkosi.default is file that tells mkosi how to build the image. We opt for a Fedora-based image here (but we might as well have used Debian here, or any other supported distribution). We need no particular packages during runtime (after all we only depend on libc), but during the build phase we need gcc and make, hence these are the only packages we list in BuildPackages=.

  5. mkosi.build is a shell script that is invoked during mkosi's build logic. All it does is invoke make and make install to build and install our little daemon, and afterwards it extends the distribution-supplied /etc/os-release file with an additional field that describes our portable service a bit.

Let's now use this to build the portable service image. For that we use the mkosi tool. It's sufficient to invoke it without parameter to build the first image: it will automatically discover mkosi.default and mkosi.build which tells it what to do. (Note that if you work on a project like this for a longer time, mkosi -if is probably the better command to use, as it that speeds up building substantially by using an incremental build mode). mkosi will download the necessary RPMs, and put them all together. It will build our little daemon inside the image and after all that's done it will output the resulting image: walkthroughd_1.raw.

Because we opted to build a GPT raw disk image in mkosi.default this file is actually a raw disk image containing a GPT partition table. You can use fdisk -l walkthroughd_1.raw to enumerate the partition table. You can also use systemd-nspawn -i walkthroughd_1.raw to explore the image quickly if you need.

Using the Portable Service Image

Now that we have a portable service image, let's see how we can attach, enable and start the service included within it.

First, let's attach the image:

# /usr/lib/systemd/portablectl attach ./walkthroughd_1.raw
(Matching unit files with prefix 'walkthroughd'.)
Created directory /etc/systemd/system/walkthroughd.service.d.
Written /etc/systemd/system/walkthroughd.service.d/20-portable.conf.
Created symlink /etc/systemd/system/walkthroughd.service.d/10-profile.conf → /usr/lib/systemd/portable/profile/default/service.conf.
Copied /etc/systemd/system/walkthroughd.service.
Created symlink /etc/portables/walkthroughd_1.raw → /home/lennart/projects/portable-walkthrough/walkthroughd_1.raw.

The command will show you exactly what is has been doing: it just copied the main service file out, and added the two drop-ins, as expected.

Let's see if the unit is now available on the host, just like a regular unit, as promised:

# systemctl status walkthroughd.service
● walkthroughd.service - A simple example service
   Loaded: loaded (/etc/systemd/system/walkthroughd.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/walkthroughd.service.d
           └─10-profile.conf, 20-portable.conf
   Active: inactive (dead)

Nice, it worked. We see that the unit file is available and that systemd correctly discovered the two drop-ins. The unit is neither enabled nor started however. Yes, attaching a portable service image doesn't imply enabling nor starting. It just means the unit files contained in the image are made available to the host. It's up to the administrator to then enable them (so that they are automatically started when needed, for example at boot), and/or start them (in case they shall run right-away).

Let's now enable and start the service in one step:

# systemctl enable --now walkthroughd.service
Created symlink /etc/systemd/system/multi-user.target.wants/walkthroughd.service → /etc/systemd/system/walkthroughd.service.

Let's check if it's running:

# systemctl status walkthroughd.service
● walkthroughd.service - A simple example service
   Loaded: loaded (/etc/systemd/system/walkthroughd.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/walkthroughd.service.d
           └─10-profile.conf, 20-portable.conf
   Active: active (running) since Wed 2018-06-27 17:55:30 CEST; 4s ago
 Main PID: 45003 (walkthroughd)
    Tasks: 1 (limit: 4915)
   Memory: 4.3M
   CGroup: /system.slice/walkthroughd.service
           └─45003 /usr/local/lib/walkthroughd/walkthroughd

Jun 27 17:55:30 sigma walkthroughd[45003]: Initializing.

Perfect! We can see that the service is now enabled and running. The daemon is running as PID 45003.

Now that we verified that all is good, let's stop, disable and detach the service again:

# systemctl disable --now walkthroughd.service
Removed /etc/systemd/system/multi-user.target.wants/walkthroughd.service.
# /usr/lib/systemd/portablectl detach ./walkthroughd_1.raw
Removed /etc/systemd/system/walkthroughd.service.
Removed /etc/systemd/system/walkthroughd.service.d/10-profile.conf.
Removed /etc/systemd/system/walkthroughd.service.d/20-portable.conf.
Removed /etc/systemd/system/walkthroughd.service.d.
Removed /etc/portables/walkthroughd_1.raw.

And finally, let's see that it's really gone:

# systemctl status walkthroughd
Unit walkthroughd.service could not be found.

Perfect! It worked!

I hope the above gets you started with Portable Services. If you have further questions, please contact our mailing list.

Further Reading

A more low-level document explaining details is shipped along with systemd.

There are also relevant manual pages: portablectl(1) and systemd-portabled(8).

For further information about mkosi see its homepage.

Read the whole story
rosskarchner
18 days ago
reply
DC-ish
Share this story
Delete

Fire Kites, Decentralized Drone Defense, and Israel's New Existential Vulnerability

1 Share

Here's a good example of global guerrilla tinkering and decentralized defense.

Fire Kites

Palestinian guerrillas in the Gaza strip have developed a simple and effective weapon for disrupting Israel.   It's a fire kite.  Here are the details (reuters video report):

  • A homemade kite, built out of transparent plastic (making it hard to spot).  This is the delivery vehicle.

Kites-Gaza-B-5.5.18

  • The warhead is a long tail and fringe soaked in flammable liquid.  The tip of the tail is a burning rag or burning coal.  

Kites

  • The kites drift into Israel (courtesy of prevailing winds along the entire border with Gaza), fall to the ground, and start fires.
Prevailing Winds over the Black Sea
Prevailing winds

So far, the kites appear to be working, largely because they are:

  • so easy and cheap to build (under $3 in commonly available materials),
  • so easy to launch (get the kite aloft from anywhere along the border and let it go), and
  • able to do significant damage when they do land (millions in fire damage already reported).

To defend against these attacks:

  • Israel turned to reservists who are drone hobbyists (this allowed them to stand up a drone defense system nearly overnight),
  • The drone pilots either tangle themselves up with the kite (with both falling to ground) or they grab what's left of the kite's string and drag it to the ground.  So far, they've downed far more than 500 fire kites this way.
  • Israel has promised to compensate the drone owners for any broken drones.  

The Real Threat to Israel

However, as interesting as these fire kites are, there is a twist.  It's likely that these fire kites are good news for Israel.  Here's why:

  • The fire kites provide Israel with a figleaf.   A visible justification for continuing aggressive military action against the Palestinians in Gaza (the kites, although expensive in property damage, are largely non-lethal and they provide dramatic visuals).  This justification provides some protection against an emerging threat to Israel.

  • Israel has always been vulnerable to a boycott.  An investment, business, and systemic boycott that disconnects Israel from the world.  So far, Israel has been good at working behind the scenes to prevent this.  However, that's not likely good enough anymore.  The world has changed.  

  • The rapid rise of the #resistance network and surging networked corporatism changes everything.  These networks have both the capability and the disposition to disconnect Israel from the world, nearly overnight, if they are triggered by perceptions of immoral behavior directed against the people of Gaza.  State directed violence, without a meaningful threat to justify it (a threat that these kites seem to provide), could be enough to trigger these networks into action.

PS:  The Fire Kites are also a good opportunity to run through the logic of making big jumps in innovation (ala "making snowmobiles" as John Boyd would say).   I'm writing up my notes on this and will post them soon.   

PPS:  Things are spinning out of control very fast now in the US.  The fictive kinship that held us together as a nation has collapsed.   Nothing but dark skies ahead.  

 

 

 

 

Read the whole story
rosskarchner
21 days ago
reply
DC-ish
Share this story
Delete

Complaint Lodged With Alcohol Board Calls Trump's 'Good Character' Into Question

1 Comment and 2 Shares
Photo by Geoff Livingston . A new complaint filed with the D.C. Alcoholic Beverage Regulation Administration calls for the agency to rescind the Trump…
Read the whole story
rosskarchner
25 days ago
reply
Ha!
DC-ish
acdha
25 days ago
reply
Washington, DC
Share this story
Delete

If a struggling company gets kicked off the Dow, then is the Dow meaningful?

1 Share

General Electric is getting kicked off the Dow because of its financial struggles. But how meaningful is the Dow if it keeps getting rid of companies when they run into trouble. Some people point out that the Dow has mostly gone up over the last 120 years, and they see this as evidence of an expanding economy. But if they hide all the bad news, and only keep the good news, is the index really meaningful? If you only allow successful companies on the Dow, and you kick them out as soon as any have trouble, then obviously the Dow will tend to be going up. But that doesn’t reflect any kind of reality about the economy. It just means the Dow keeps adding successful companies to its index.

From the article:

This isn’t GE’s first rejection letter. The company was removed from the Dow index in 1898, only to return seven months later. It was replaced again in 1901, but came back for good in 1907. This latest ouster is a sign of the company’s financial struggles, and of an ongoing shift (paywall) in which types of companies are still considered key indicators of the state of US industry.

“General Electric was an original member of the DJIA in 1896 and a member continuously since 1907,” David Blitzer, managing director of the S&P Dow Jones Indices committee, said in a statement (pdf). “Since then the U.S. economy has changed: Consumer, finance, health care and technology companies are more prominent today and the relative importance of industrial companies is less.”

Analysts have been speculating for months that GE might get the boot, with Deutsche Bank predicting in January that the company’s standing would be impacted by challenges around “earnings and cash pressure, tough global power generation markets, aggressive downsizing, shrinking its portfolio, management shake-up and SEC investigations.” Wrote Deutsche Bank analyst John Inch: “Apart from GE’s other challenges, as the company’s absolute share price has continued to drop, GE increasingly falls into the category of outlier and consequently a likely candidate for removal.”

Read the whole story
rosskarchner
26 days ago
reply
DC-ish
Share this story
Delete
Next Page of Stories