NetLinux From Scratch
Introduction
Build a new distro based on self-extract binaries for deployment.
This is using the "Self-Extracting tar files" as a starting point.
From the outset it will be a live CD deployment starting with the most basic wrappers.
Including an automated remastering system to build a new latest distro immediately from the upstream sources.
This will hopefully remove the need for me to actually maintain the distribution, since it will be done by the user.
Not sure if it is possible to do all this, but I'm willing to try.
The use case is my current customer ST Microelectronics
who seem to be stuck in a rut with the deployment of ST Linux
Lending heavily from the Linux from scratch project, hopefully with full documentation for the seriously uninitiated
Working almost entirely in VirtualBox for image deployment
To make it accessible to the widest possible user base
I do intend to mirror all the relevant sites so all the downloads will be available from this server.
Building a Live CD
The live CD is the same as a standard bootable ISO, which just runs more of the standard operating system
than the previous "minimal" environment to install.
This will include a compressed filesystem (squashfs), which can be directly mounted read-only
and a stackable filesystem (unionfs or aufs).
The stackable filesystem gives you the ability to write the differences from the read-only compressed system
somewhere else (usually ram) and make the operating system think it's in a standard hard disc installation.
In order to do this you need to set up the root filesystem in a special way to include all this.
You also usually need to slightly modify the way the operating system works to account for it not being on a hard disc.
An example would be that you can't reboot the operating system to make new changes take effect.
We also need the ability to create the ISO and make it bootable.
This is done with mkisofs (Which is part of cdrecord) and syslinux.
Step 1: Preparing
First download and install VirtualBox.
This will be needed to test the ISO, but we start with it here since it may be needed for the build environment too.
Download and install the latest version from the VirtualBox Downloads page.
If you are using Linux then you may be able to use your package manager to do that for you.
NOTE: If you are already using Ubuntu then you can skip over the next bits to "In Ubuntu"
For those of you not using Ubuntu on your PC (or Mac) you will need to download a version of Ubuntu and run it in VirtualBox.
You can use any version, but I would suggest Xubuntu, since it is lightweight on resources
Download the ISO image from a mirror on the Get Xubuntu page
Make sure you get the "desktop" version, not "alternate".
Next create a VirtualBox virtual machine using the instructions from "Creating your first virtual machine"
These are very simple as the wizard guides you through the process.
Make sure you select Operating System = "Linux" and "Version" = Ubuntu and give it a name (e.g. "NLFS testing"):
I would use at least 512Mb RAM for your Xubuntu live enviromnent.
There is no need to create a virtual hard disk as you will be using the host PC for storage.
VirtualBox asks you to verify you don't want one.
Just continue then finish as usual.
You will need to select the downloaded CD by changing the settings of your newly created machine
You will also need 2 CD devices since one will be permanently tied up with the Live CD.
NOTE: Ensure the live disc is the first CD device.
More information on this is in the "Storage settings" page.
Once in the booted live disc you can share a folder (e.g. you desktop folder) into the virtual enviroment.
There are a few steps to get this to work.
These are run in a terminal in the virtual machine which you can get from "Applications" (top left) -> "Accessories" -> "Terminal"
sudo -i
cd /home/ubuntu/Desktop
Firstly you need the virtual machine to connect to the internet.
This should be working already,
but if it is not check out "Ubuntu Networking for Basic and Advanced Users"
Next, since most businesses have some sort of proxy running, you will need to set that up too.
This is simply setting up some environment variables:
export http_proxy=http://<server>:<port>
export ftp_proxy=http://<server>:<port>
export https_proxy=http://<server>:<port>
or if the proxy requires authentication:
export http_proxy=http://<user>:<password>@<server>:<port>
export ftp_proxy=http://<user>:<password>@<server>:<port>
export https_proxy=http://<user>:<password>@<server>:<port>
If you need more help on this there is plenty of information on the internet.
NOTE: you will now need to share a folder using the tool on the machine interface bottom right.
More information is on the "Shared folders" page.
In the example mount below I created a shared folder and called it "Desktop".
This was mounted onto a folder called "nlfs" on the virtual machines desktop.
apt-get update
apt-get install -y virtualbox-ose-guest.*
modprobe vboxsf
mkdir nlfs
mount -t vboxsf Desktop nlfs
This should now give you access to a folder on your PC's desktop from the Ubuntu machine.
Your contents will obviously be different, but you can see the shared access with the host desktop PC.
As a minor point, to save you having to repeat these steps everytime you start your PC,
you can save the virtual machine state before shutting down your PC.
When you next boot up you simply restore the machine and you should be back to where you left off.
So, you now have a working ubuntu environment in which to proceed with the build.
What now follows is pretty much the same whether you are running Ubuntu in a virtual machine,
or have a native installation.
The next thing we need is a filesystem from which to create the disc ISO image.
As with all Linux installations, this will require a kernel, ramdisc and bootloader.
We can get the kernel and ramdisc from kernel.org (Click "Latest Stable Kernel").
Create a folder in which to work ("nlfs" in the example) and download this to it.
If you are working in a VM you can download it from your PC browser to your VM shared folder
or your Desktop if that is what is shared.
Now in Ubuntu open a terminal and change to that folder, e.g.:
ubuntu@ubuntu:~$ cd Desktop/nlfs
ubuntu@ubuntu:~/Desktop/nlfs$ tar -xjf linux*
ubuntu@ubuntu:~/Desktop/nlfs$ make distclean defconfig vmlinux modules bzImage
. . .
CC arch/x86/boot/video-vga.o
CC arch/x86/boot/video-vesa.o
CC arch/x86/boot/video-bios.o
LD arch/x86/boot/setup.elf
OBJCOPY arch/x86/boot/setup.bin
OBJCOPY arch/x86/boot/vmlinux.bin
HOSTCC arch/x86/boot/tools/build
BUILD arch/x86/boot/bzImage
Root device is (8, 5)
Setup is 13436 bytes (padded to 13824 bytes).
System is 3957 kB
CRC 70ddbaae
Kernel: arch/x86/boot/bzImage is ready (#2)
ubuntu@ubuntu:~/Desktop/nlfs$
This may take a while, especially if working in a VM.
Development
graham@graham-workstation:~/Desktop/sda7/nlfs$ rm -vfr busybox* && wget http://www.busybox.net/downloads/busybox-1.17.4.tar.bz2 && tar -xjf busybox* && pushd busybox* && make clean defconfig busybox; popd; ./busybox-1.17.4/busybox hostname
## -- dynamic linking: use wget http://www.busybox.net/downloads/binaries/1.17.2/busybox-i686
## -- Added to arch/x86/boot/Makefile :
isoimage: $(obj)/bzImage
-rm -rf $(obj)/isoimage
mkdir $(obj)/isoimage
cp -v /home/graham/Desktop/Archive/nlfs/busybox-i686 $(obj)/isoimage/busybox
ln -s busybox $(obj)/isoimage/sh
for i in lib lib64 share end ; do
## -- Then ran
make isoimage FDARGS="root=/dev/sr0 init=/sh debug"
getandbuild.sh:
#!/bin/bash
rm -vfr busybox-*
wget http://www.busybox.net/downloads/busybox-1.17.4.tar.bz2
tar -xvvjf busybox-1.17.4.tar.bz2
cp -v busybox.config busybox-1.17.4/.config
pushd busybox-1.17.4
make clean busybox
popd
rm -vfr linux*
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.36.1.tar.bz2
tar -xvvjf linux-2.6.36.1.tar.bz2
pushd linux-2.6.36.1
make distclean defconfig vmlinux modules bzImage
popd
./isoimage.sh linux-2.6.36.1/arch/x86/boot/bzImage busybox-1.17.4/busybox
isoimage.sh:
#!/bin/bash
KERNEL="$1"
BUSYBOX="$2"
FDARGS="root=/dev/sr0 init=/init.sh debug"
DATETIME="`date +%y%m%d%H%M%S`"
IMAGE_CMDLINE="default linux $FDARGS"
ISOFOLDER=isoimage
ISOLINK=FemtoNetLinux.iso
echo "KERNEL=$KERNEL"
echo "BUSYBOX=$BUSYBOX"
echo "FDARGS=$FDARGS"
echo "DATETIME=$DATETIME"
echo "IMAGE_CMDLINE=$IMAGE_CMDLINE"
echo "ISOFOLDER=$ISOFOLDER"
echo "ISOLINK=$ISOLINK"
if [ ! -f "$KERNEL" ]
then
echo "Kernel not found."
exit 2
fi
if [ ! -f "$BUSYBOX" ]
then
echo "Busybox not found."
exit 3
fi
rm -vrf $ISOFOLDER
mkdir $ISOFOLDER
cp -v "$BUSYBOX" $ISOFOLDER/busybox
ln -vs busybox $ISOFOLDER/sh
for i in lib lib64 share end
do
echo "i=$i"
ls -lh /usr/$i/syslinux/isolinux.bin
if [ -f /usr/$i/syslinux/isolinux.bin ]
then
cp -v /usr/$i/syslinux/isolinux.bin isoimage/isolinux.bin
break
fi
if [ $i == end ]
then
echo "Can't find isolinux.bin"
exit 1
fi
done
cp -v "$KERNEL" $ISOFOLDER/linux
echo "$IMAGE_CMDLINE" > $ISOFOLDER/isolinux.cfg
for MOUNTPOINT in bin sbin etc dev proc init sys mnt tmp usr var
do
mkdir -vp $ISOFOLDER/$MOUNTPOINT
done
cp -v init.sh $ISOFOLDER/
mkisofs -J -r -o "$DATETIME".iso -b isolinux.bin -c boot.cat \
-no-emul-boot -boot-load-size 4 -boot-info-table \
$ISOFOLDER
isohybrid "$DATETIME".iso
rm -vrf $ISOFOLDER
rm -vf $ISOLINK
ln -vs "$DATETIME".iso $ISOLINK
init.sh:
#!/busybox sh
echo
echo "=============== Hello from Femto NetLinux (www.netliux.org.uk) ================="
mount -t proc none /proc
mount -t sysfs none /sys
for MOUNT in bin sbin etc dev tmp usr var
do
mount -t tmpfs tmpfs /$MOUNT
done
mdev -s
mount
ln -s /proc/mounts /etc/mtab
mkdir -p /var/run /var/log
ls -lR /var
syslogd
cat /var/log/messages
ls -Rl /dev
echo '#!/busybox sh
echo $@
if [ $1 != "bound" ]
then
exit
fi
env
ifconfig $interface $ip $netmask
route add default dev $interface gw $router
echo "nameserver $dns" >/etc/resolv.conf
sleep 1
ping -c1 $dns
' >/var/udhcpc.sh
chmod a+x /var/udhcpc.sh
if ifconfig eth1 up
then
udhcpc -s /var/udhcpc.sh -S -i eth1
fi
if ifconfig eth0 up
then
udhcpc -s /var/udhcpc.sh -S -i eth0
fi
ls -lR /etc
echo "127.0.0.1 localhost" >/etc/hosts
mkdir -p /var/www
mkdir -p /var/www/cgi-bin
echo '<HTML>
<HEAD>
<TITLE>Femto Netlinux</TITLE>
</HEAD>
<BODY STYLE="text-align:center">
<H1>Hello from Femto Netlinux</H1>
<A HREF="http://www.netlinux.org.uk">NetLinux Home</A><BR>
<A HREF="/cgi-bin/env.sh">CGI test</A><BR>
</BODY>
</HTML>
' >/var/www/index.html
echo '#!/busybox sh
echo "Content-type: text/plain"
echo
env
' >/var/www/cgi-bin/env.sh
chmod a+x /var/www/cgi-bin/env.sh
httpd -h /var/www
openvt -w -c 1 /busybox sh
busybox.config:
Boot this in a virtual machine or on a real PC and you get a prompt in under 5 seconds.
Gives networking, but name resolution doesn't work.
Could be used as a rescue disc.
Type /busybox for available commands
As above but with a Femto web server.
Brings up eth0 and eth1 so assuming you have two interfaces on your VM one for NAT and one host-only
You can view in a browser via the host-only
Initially heading towards something similar to Tiny Core Linux.
Also check Minimal Linux and Create a Custom Live Linux CD - Leveraging BusyBox and OpenSSH
