Recent searches
Search options
Administered by:
This week, I'm again starting my #SysAdmin class where we focus on a lot of practical aspects commonly missing in Computer Science curricula. I'm running the same class as an internal study group at work as "Internet Operations".
The syllabus and all course info is available here:
https://stevens.netmeister.org/615/
All video lectures are public and available for free on YouTube:
https://www.youtube.com/@cs615asa/videos
Follow along! I'll be posting weekly links in this thread throughout the semester.
System Administration
Week 1, Introduction
We cover all the meta info for the course, how System Administration fits into a Computer Science curriculum, our syllabus, systems used, grading policy etc. (There will be Venn diagrams...)
System Administration
Week 1, The Job of a System Administrator
In this video, we try to show what things SysAdmins may encounter in their day to day routine, ranging from power tools to actual and symbolic duct tape and WD40. With this, we try to sort out just how the job may be defined. It's not that easy, as it turns out...
System Administration
Week 1, Core Principles
In this video, we present a few core principles that will guide us throughout the semester, including our three pillars: Scalability, Security, and Simplicity. We'll also get to know a few basic "laws", well known by any System Administrator. If you're wondering what all this has to do with Legos, please tune in...
System Administration
Week 1, #Unix History
Exhibiting one of the cardinal virtues of System Administrators — laziness — we're reusing a video from my #programming class to give a whirlwind history tour: Bell Labs, Berkeley & the Unix Wars, USL v. BSDi, the birth of #NetBSD & #FreeBSD, the development of GNU & its adoption of #Linux, all leading to Unix on your fridge, car, and mobile phone.
Last semester's #apue thread:
https://mstdn.social/@jschauma/113087999769442250
System Administration
Week 1, AWS Aliases
In this video, we demonstrate the use of shell aliases and functions to save ourselves some typing whenever we run #AWS #EC2 commands.
To ensure we get a full dual-stack IPv4 / #IPv6 environment, we're also following this guide:
https://www.netmeister.org/blog/ec2-ipv6.html
The functions shown in the video make use of the subnet and security group described in the blog post and are available here:
https://github.com/jschauma/cloud-functions/blob/main/awsfuncs
System Administration
Week 2, Storage Models and Disks
This week is all about filesystems and storage. We start with a discussion of conceptual storage models, such as Direct Attached Storage (DAS), Network Attached Storage (NAS), Storage Area Networks (SANs), and Cloud Storage.
System Administration
Week 2, Devices and Interfaces
In this video, we'll talk a bit about storage devices and their interfaces, including SCSI, ATA, SSD, Fibre Channel, and hinting at storage configurations like JBOD and RAID.
System Administration
Week 2, Storage Virtualization
In this video, we cover the concept of storage virtualization -- combining individual disks into larger storage pools and utilizing resources from such a pool. This includes a discussion of RAID and some of the different supported levels as well as Logical Volume Management (by example on #Debian #Linux). We also illustrate some of these properties by example of ZFS (on #OmniOS).
System Administration
Week 2, Physical Disk Structure
In this video, we'll take a quick look at what an old-fashioned spinning-magnetic-platters hard disk drive actually looks like. This will help us understand the underlying addressing schemes such as CHS and LBA as well as what physical aspects affect hard disk performance as well as partitioning requirements.
System Administration
Week 2, Partitions
In this video, we'll talk about how to divide a single disk -- physical or virtual -- and how the partitions relate to the physical structure of the disk. We show examples partitioning disks on #NetBSD, #OmniOS, and #Linux using the disklabel, fdisk, and format tools.
System Administration
Week 3, The Boot Process & the MBR
In this video, we discuss the boot process on a high level as well as take a fairly detailed look at the MBR. We'll create a suitable #NetBSD BIOS partition by hand, utilizing the dd(1) command because using fdisk(8) would be just too easy. In the process, we learn a fair bit about the structure of the boot sector.
(I really should do an update of this with UEFI/GPT...)
System Administration
Week 3, Filesystems
In this video, we pretend to be a filesystem, trying to store all our cat photos in a reasonable manner on a raw disk by manually writing bytes to the raw block device. We also show how the tar(1) utility creates output that very much resembles a filesystem format. (There's a reason for that...)
System Administration
Week 3, The Unix Filesystem
In this video, we look at the traditional #UNIX Filesystem and how that is structured. We use the newfs(8) tool to create such a filesystem (well, an FFS, anyway), and the dumpfs(8) tool to inspect it.
For a slightly deeper look at the UNIX Filesystem, take a look at this video from my "Advanced #Programming in the UNIX Environment" lecture series:
System Administration
Week 3, Files go hier(7)
In this video, we're wrapping up our discussion of file systems and partitions with a look at file types (e.g., directories 
, fifos ↴↱, character devices 
, block devices 
, sockets 
, hard links 
, symbolic links →) as well as the file systems mounted by default. We close with a look at the filesystem hierarchy as defined in the hier(7) manual page.
System Administration
Week 3, Resizing file systems
Closing out our week of discussing the Dark Arts[1] of file systems, we show how to resize an existing file system on #NetBSD using the resize_ffs(8) tool. We first increase the size of a 512MB partition to 1GB, then shrink it down to 256MB:
For comparison, we then repeat the same exercise on #Debian #Linux, using resize2fs(8):
System Administration
Week 3, Additional Reading
GUID Partition Table (GPT)
https://uefi.org/specs/UEFI/2.10/05_GUID_Partition_Table_Format.html
The Security War in File Systems: An Empirical Study from A Vulnerability-centric Perspective
https://dl.acm.org/doi/10.1145/3606020
And to dive a bit deeper into #ext4 and FFS.
Understanding ext4 Disk Layout:
https://blogs.oracle.com/linux/post/understanding-ext4-disk-layout-part-1
ext4 Data Structures and Algorithms:
https://www.kernel.org/doc/html/latest/filesystems/ext4/
A Brief History of the BSD Fast Filesystem:
https://freebsdfoundation.org/wp-content/uploads/2016/04/A-Brief-History-of-the-BSD-Fast-Filesystem.pdf
System Administration
Week 4, Types of Software
With this video, we begin our topic of "software": what types of software there are, how they fit together, how to install software, and how to manage dependencies. We try to draw a terrible analogy to - what else - cars, and quickly realize that the distinctions between firmware, operating system, system software, add-on software are difficult to make.
System Administration
Week 4, OS Installation
In this video, we run through the manual installation of #NetBSD onto a virtual machine to illustrate the details of the process from partitioning and mounting the disk, extracting the sets, installing the bootloader, creating device nodes to updating /etc/rc.conf.
We also discuss planning of the OS installation by looking at data classification into shareable/non-shareable and static/variable data.
System Administration
Week 4, Package Management
In this video, we continue our discussion of the difference between the operating system and so-called "add-on software". We conclude that in order to install and maintain all such software, we want to use a package manager, and illustrate common features by example of the 'dpkg', 'rpm', and #NetBSD's #pkgsrc tools.
System Administration
Week 4, Package Manager Pitfalls
In this video, we discuss some of the problems with package managers, native language-specific packaging solutions, and the implications of their use on dependency resolution, package integrity, and trust. We revisit the "left-pad" (2016) and "dependency confusion" (2021) incidents to illustrate some of these problems.
System Administration
Week 4, Multiuser Fundamentals
We don't have a video for this segment, but here are the slides discussing group dynamics, foundations of trust (and #ZeroTrust), like Least Privilege, headless users or service accounts, group permissions, many examples of different forms of multi-factor authentication #mfa, and raising privileges (a topic we'll come back to again later in the semester).
System Administration
Week 5, Networking I: Layers
In this video, we begin our longer discussion on the topic of "networking". We're using tcpdump(8) to capture a single TCP SYN packet and start looking at the MAC and IP information, teasing out each individual byte. And yes, we will mention those OSI layers...
System Administration
Week 5, Networking I: IPv4 Basics & CIDR subnetting
In this video, we cover the basics of the 32-bit IPv4 address and how we organize networks using Classless Inter-Domain Routing or CIDR subnetting. (Don't worry, #IPv6 comes in our next video.)
System Administration
Week 5, Networking I: #IPv6 Basics
In this video, we get familiar with our Big Hero IPv6, looking at the structure of the IPv6 header and IPv6 address representations.
System Administration
Week 5, Networking I: IP Allocation & IPv4 Exhaustion
Mommy, where do IP addresses come from? We discuss how IANA allocates IP addresses to the Regional Internet Registries and try to illustrate just how large the #IPv6 address space is.
System Administration
Week 5, Networking I: The Physical Internet
In this video, we look at the physical structure of the internet, with a focus on submarine internet communications cables. Jumping from the bottom of the OSI stack all the way to Layer 9 ("political"), we then discuss how different countries use their political power to enforce internet blocks on their citizens, leading us to warrantless wiretapping in AT&T's room 641A.
System Administration
Week 5, Networking I: A Network of Networks
In this video, we look at how independent networks connect to one another, how Autonomous Systems numbers allow us to identify network operators, and how peering between independent ASs works.
System Administration
Week 6, Networking II: A simple request
In this video, we trace a simple HTTP request made via telnet to find out just how exactly our application knows how to connect to the remote server. In the process we learn about the ktrace(1) utility (strace(1) on #Linux), as well as the nsswitch.conf(5), hosts(5), and resolve.conf(5) configuration files.
System Administration
Week 6, Networking II: A simple request II
In this video, we pick up our tcpdump output collected in the previous video and begin to analyze the packets we find in there. We observe our AWS instance being scanned by some random host on the internet, see the ARP requests and replies, our DNS query, and the TCP packets for our HTTP request, yielding an illustration how a simple request causes traffic across multiple layers.
System Administration
Week 6, Networking II: ARP and NDP
In this video, we illustrate the functionality of the Address Resolution Protocol (ARP) and it's #IPv6 equivalent, the Neighbor Discovery Protocol (NDP).
@jschauma excellent title 
LOL.
i first learned networking when every govt was pushing OSI as the "future". even had a job doing OSI protocol conformance testing (thankfully i did layer 4 stuff). even i'd be hard pressed to explain how the abstractions for layers 5&6 have working examples in the current network. it's really mostly kinda sorta rammed into the app, if those functions are done at all.
when asked the significance of the 7 layer model, my answer is usually "it's a highly used footnote in older networking textbooks". :)
@paul_ipv6 @jschauma Occasionally bits of the layers pop out from their tomb to scare us.
Portions of the old OSI session layer exist in SCTP and, amusingly enough, QUIC. Mostly in the sense of "streams" but never the old framing layer stuff that would be nice for protocol developers.
Presentation layer tends to be ASN.1 and perhaps you can squint at it with very bad glasses and visualize YANG models in there....
@paul_ipv6 Nice. I primarily use the OSI model to reference the hidden layers. https://ops-lessons.creator-spring.com/listing/osi-9-layer-model?product=658
@paul_ipv6 @jschauma I'm pretty sure I remember a quote from Michael A Padlipsky:
If you know what you're doing three layers is enough, if you don't, seventeen layers won't help.
https://en.wikipedia.org/wiki/Michael_A._Padlipsky?wprov=sfla1
Fun fact for your students: pretty much every interview I have ever given (and all the good ones that I have been on the other end of) has featured some variant on this week's topic as an open-ended question.
@dashdsrdash yep, yep, very popular :-)