HOW-TO: Boot to Recovery/Rescue Mode

Booting to Recovery mode or Rescue mode is important. This is primarily due to the single most important reason -- nobody can predict an impending disaster. And this is why infrastructure personnel like me have jobs.

Below I have a screenshot of one of the many reasons that you might want to boot in Recovery mode. The error being "VFS: Unable to mount rootfs on unknown-block(0,0)". The cause for this error is another topic altogether, but this error needs the intervention of a system administrator.

VFS: Unable to mount rootfs

The error can manifest on a baremetal set-up or a virtual machine. The screenshot is on an Ubuntu LTS 16.04 guest running on VMware ESXi v5.5. The guest operating system simply won't boot no matter what.

First restart and interrupt the boot process and modify some boot options. Look for the line containing "linux" and at the end of the line, append "systemd.unit=recovery.target" (NOTE: Both rescue.target and recovery.target works). Then press F10 to boot.

systemd.unit=recovery.target

Modifications above will interrupt the boot process to allow recovery mode of other installed kernel images, otherwise not shown on the boot menu. When presented with that menu, select a recent kernel image a few versions down from the current unbootable one. Make sure to select the option with "(recovery mode)" in it. Otherwise, just input the root password when on the recovery console when prompted.

On the console terminal, remove the unbootable kernel image (or force a reinstall), or execute other applicable steps/procedures to successfully boot the system. Hope this tip helps in accessing the recovery mode or rescue mode the next time a need for it arises.

RELATED: Quantifiably Measure Boot Time Optimizations

Do you have another way to access the recovery mode? Please share it on the comments below.

TWEAK: Customized Right-Click Menu

I often get asked by colleagues how I setup a customized right-click menu for my Desktop. Don't get me wrong, I still use the notebook that the company issues but I ask for limited administrative privileges to be able to customize the notebook to my working style as I like to access my often used software on the right-click menu. It is a quick access menu and you will see how it looks in a bit.

Other than this "menu", I also leave my desktop with very minimal contents, icons, or shortcuts. This makes access my Desktop quick and very responsive. If you feel like your computer is bogging down, try removing stuff from your Desktop (trust me, it goes a long way).

For you to have an idea what I'm talking about, here's a screenshot of my Desktop.

Right-Click Custom Menu

The question then is: How did I add several lines to the right-click menu? Good question, read on as I will outline how this gets done.

WARNING! This involves making changes to the registry and could potentially end up in disaster. Proceed at your own risk!

Now that you have ample warning, open the registry editor.. Head down to "HKEY_CLASSES_ROOT\Directory\Background\shell" or "HKCR\Directory\Background\shell". This requires Administrator privileges, so if you don't have it on your notebook, ask for it first. Otherwise, this will not work for you.

Right-click on "shell", and select "New" --> "Key". Name this key with what you want to appear on the right-click menu. In my case, I used the key "anaconda update" specifically for updating my anaconda3 install in my notebook.

Next, right-click on the newly created and key create another key. This time name it "command" (do not use any other name). On the data pane, fill up the "(Default)" field with the command used to run the created key. In my example above, commands specific to "anaconda update" -- "conda update --all --yes".

You should have something similar to the screenshot below. I already have several entries in my Desktop so the existing Keys on your registry might appear different. In my set-up, I also added another key to launch "jupyter notebook" with the same command among several additions.

Regedit HKCR\Directory\Background\shell

The change takes effect immediately, so check by right-clicking your Desktop and see the menu appear. If it doesn't appear, re-check the steps executed.

RELATED: TOO MANY USER/GDI objects are being used..

This setup has also helped me a lot in uncluttering my Desktop from icons/shortcuts that would slow it down. Not to mention, give it a very simple, clean look.

TWEAK: Quantifiably Measure Boot Time Optimizations

If there is a word that first comes to mind when running virtual machines (VM), I would say "lean". Meaning, if the VM does not need to run a certain service, then, disable it; ensure that only the services required should be running. This will not only improve utilization but it does free up resources, thereby minimizing the probability of contention. If you are using cloud services, the pay-per-use model pays out the leaner the virtual machine.

On the flip side, knowing which services really do contribute to a big reduction to wastage is key. The question then: How do you quantifiably measure which optimizations give the most bang for your buck? Read on..

One thing to note, this tweak will only work on modern Linux operating systems with systemd. The old-school "init"-run systems will not be able to run the command, as the required binaries will not be present.

The above statement is the give-away to the command used. In order to do this, execute the command "sudo systemd-analyze plot > some-file.svg" on a terminal window. Then open the resulting output file, in this case "some-file.svg" in your browser. Prior to making tweaks, take a baseline measurement and see where optimizations can be applied. Subsequently, after another tweak, run the same command again to measure and compare the results with the baseline.

The added benefit of executing this procedure is that, you also get an idea of the time it takes to boot-up your virtual machine.

systemd-analyze plot

Having done this baseline measurement in my virtual machine, I now know that it takes ~16seconds to boot-up. And that, the postfix service is taking its time to start-up causing my virtual machine a few seconds delay to completely boot. Understandably, mysql.service takes a while to start causing an additional ~7secs to boot time. I really cannot disable that service but I could optimize elsewhere.

If I really don't require postfix running there. It would shave off ~1sec off the boot time. But really, ~16secs boot time is not bad.

Out of curiosity, I ran the same command in my AWS EC2 instance. The screenshot is below.

AWS EC2 systemd-analyze

It is amazing to see that the EC2 instance, with a MySQL database running, only takes ~8secs to boot-up. Now that is lean or what!

RELATED: P2V (Physical to Virtual) Prep Work for Ubuntu

Go ahead and see if you can make tweaks in your environment. Let me know if this has helped by your comments and feedback.

INFO: Process audiodg.exe Consumes High Memory

A recent not-so-good experience led me to write about it.. Have you ever been in the "flow" and so absorbed in your work that when you get interrupted, you find it very difficult to get back to where you were at? I recently had that, and the disruption came from a technology failure, not a human interaction. I will get to the details next.

I was working on a large dataset in my notebook and all of a sudden I got an OOM notification (out of memory). The prompt was similar to "Your computer is running low on memory".. I could not get a screenshot of the error as I could not do anything more after that and was forced to press the power button long enough to restart.

After reboot, I tried to replicate what I was doing previously and launched the Task Manager to see if it has clues to the issue. What I saw almost pushed me off my chair. The executable audiodg.exe consumed 1.3GB of memory!

Although I have 16GB of memory to play with, I knew something is not right. Now that I was able to take a screenshot to share, you will find it below. It shows a third of system memory already consumed.

AUDIODG.EXE 1.2GB Memory

The executable audiodg.exe is an integral part of the Windows 7 Operating System. It is the "Windows Audio Device Graph Isolation" and if you terminate the process you lose audio output on your notebook, as in my experience.

Given my encounter with low memory and eventual unresponsive system, I would not want to experience that same scenario again and need to resolve it as much as I can. So I tried several suggestions from the web -- check for virus, check the digital signature of the file, etc.. And soon enough a solution was found.

It is worth noting that even on a fresh reboot, the process audiodg.exe still consumes about the same amount of memory. So memory leaking was scratched out as a possible suspect.

The culprit was installing an updated driver to the audio card of my notebook. After rollback to the previous driver, the problem went away. So I have this advise to folks out there who follow my posts: If you have a software running on your system that watches out for outdated drivers and you get notified each time a version is available, throw it away. Uninstall it, as it is useless. Just replace the driver if it causes you problems, otherwise things run fine and there is no need to update.

RELATED: Confirm Free Memory Slots on your Server (Linux)

This would sound like a cliche, but "if it ain't broken, don't fix it!" Trust me, there is truth to this advise. With that, let me leave you this question: How much memory does audiodg.exe consume on your computer?

TIP: Protect Your Router Against IP Spoofing

I was reading through some of the best practice configuration of Mikrotik routers, and routers in general. I found a very helpful tip about helping keep the internet secure for others. And this is related to the topic stated in the title -- IP Spoofing. In order to gain more information regarding this terminology, here's a resource that defined it clearly. Also quoted the same definition below.
IP Spoofing is a technique used to gain unauthorized access to machines, whereby an attacker illicitly impersonate another machine by manipulating IP packets. IP Spoofing involves modifying the packet header with a forged (spoofed) source IP address, a checksum, and the order value.

As a responsible infrastructure administrator, I am obliged to contribute to the security of others. And with that, likewise share the same information on how to do it as well for others benefit. If you own a Mikrotik router, follow the procedure below.

RP Filter. The "RP Filter" is used for packet source validation. As defined in RFC3704 Abstract, it is designed to limit the impact of distributed denial of service attacks, by denying traffic with spoofed addresses access to the network, and to help ensure that traffic is traceable to its correct source network.

To be better informed about this change, here's a link to the Mikrotik Wiki.

So how is this done in the Mikrotik?

To those who want to do it via command line interface (CLI), execute this using a privileged account:
/ip settings
set rp-filter=strict

Those who would rather use WinBox utility, IP » Settings. Set "RP Filter" to "strict".

Winbox IP Settings

In the screenshot above, "TCP SynCookies" is ticked. This is a further router protection from DDoS via SYN floods. For the CLI equivalent to that, the CLI command above should be changed to:
/ip settings
set rp-filter=strict tcp-syncookies=yes

RELATED: Mikrotik Articles in This Series

The above changes did not increase the resource consumption on my router. Monitor your router for changes to utilization of resources to see if the change does introduce significant load.