Tutorial #20: Home (Network) Improvement using a Pi


Update #1 (Aug 19, 2018):

With the passage of time, things change and in this case it is mostly for the better. I was setting up the Pi once again and was pleasantly surprised that some of the workarounds mentioned previously are no longer needed. Thus, I have edited the original tutorial to accommodate these changes while striking out the old text.

Also, after comparison, I found AB-Solution to be a better solution for network-wide ad blocking, if you happen to have a Merlin supported Asus router like I do. Since the router is on 24x7, having an old ext2 formatted pen drive plugged in to the router itself and running ab-solution is a better alternative as against running the Pi 24x7. It also has various preset ad blocking files that suit different needs while pixelserv-tls does a great job with HTTPS ads.

Lastly, I failed to mention the option of having a Samba server running on the Pi itself in order to access the files directly from the pen drive over the network. This can be accomplished by simply following the official tutorial on the Raspberry Pi website.

Original Article (May 3, 2018):

I had previously shared some tutorials in setting up the Pi and putting it to good use. However, the use cases I mentioned then have ceased to exist. The Fire TV has taken care of most of my multimedia needs and I have come to realise that I really don't have much time to go back in time for nostalgia. For the retro needs that remain, the lesser-used Core M 5Y10 equipped Windows tablet of mine does a much better job plugged in to the TV.

Hence, it is time to put the Pi to good use in a different sense. Thankfully, the versatility of the Pi means that it is not difficult to identify its next project. Ads can become a nuisance, especially for the more aged members of the family and hence my first intent was to set up an ad blocker across my home network. However, putting the Pi to such limited use and keeping it on 24/7 would be quite a waste, so I decided to also repurpose it as a download box with centralised storage.

Setting up the tools necessary to accomplish these tasks seemed straightforward. However, the relevancy of publicly available tutorials diminish over time due to changes in technology. In fact, I had to put quite some effort beyond the listed tutorials and hence I have decided to put the same to words for posterity.

Before starting out:

PINN is a great utility when multi-booting across different distributions on the same SD card. However, Raspbian alone fits the bill for the current use case. Hence, writing the raw Raspbian image directly on the card is preferable as it provides more usable space. As far as writing on the card goes, Etcher is the way to go.

1. Pi-hole®

As the website so prominently displays, all you need is a single command.

curl -sSL https://install.pi-hole.net | bash
However, I made a couple of settings that are worth mentioning:

a. The predefined list of upstream DNS providers does not yet include Cloudflare which I found to be the fastest of the lot. Hence, it would be worthwhile to use the custom option and enter the Cloudflare DNS Server IP addresses of 1.1.1.1 and 1.0.0.1.

b. The other part of the equation is setting up the home equipment to use the local DNS server. In case of Asus routers, this implies changing the DNS Server IP address to the Pi-hole one, not only on the WAN page (under WAN DNS Settings) but also under LAN > DHCP Server > DNS and WINS Server Setting. Make sure that there are no other IP addresses present in either of the pages. You could also run the DHCP server on Pi-hole, in which case the latter setting is not needed. However, since I use the router-assigned IP addresses for other functions (eg. VPN), I prefer to have the DHCP server running on the router itself.

2. qBittorrent

qBittorrent has been my preferred Bittorrent client for quite some time with it being open-source and having proper support for proxies as against Transmission. It can be installed easily using APT, though I prefer the headless route.

sudo apt-get install qbittorrent-nox
My primary endeavour was to have the downloaded files ready on the USB 3.0 hard disk connected to my router (and thus acting like a NAS) while minimising the read-write operations. Since it is not a great idea to write to the SD card running the client, I decided to plug in an old 32 GB pen drive to act as the "working folder" by adding it under Downloads > Hard Disk > Keep incomplete torrents in:

The next part was to add the network drive as the final resting place by entering its address under  Downloads > Hard Disk >  Save files to location: and also under the Copy .torrent files for finished downloads to field. The latter is sometimes necessary due to some quirks in different versions of qBittorrent. The external network drive needs to be mounted within Raspbian on boot which can be accomplished by editing /etc/fstab with these details:

mount -t cifs //xx.xx.xx.xx/folder /media/NAS -o rw,vers=2.0,username=abc,password=xyz
//xx.xx.xx.xx/folder /media/NAS cifs vers=1.0,username=abc,password=xyz,x-systemd.automount

where,
xx.xx.xx.xx -> LAN IP address as configured on the router running the Samba server
folder -> Path to the folder on the network drive that needs to be mounted
/media/NAS -> Path on the Pi where the network drive is to be mounted

In my case, I had to specifically mention the SMB version (2.0), without which the mounting would fail as well as the rw argument to be able to write to the device. Also. it is a good idea to update the cifs-util package from APT prior to editing the fstab file, as mentioned above.

Note: The earlier entry no longer worked with the June 2018 version of Raspbian due to which I had to use the alternative entry in /etc/fstab mentioned above. Also, I was only able to get v1.0 working this time despite the server supporting v2.0 as well.

Finally, to cover for unexpected reboots, it is preferable to have qBittorrent autostart which can be accomplished using systemd. First, create the startup script using:

sudo nano /etc/systemd/system/qbittorrent.service
Next, enter its contents as follows:

[Unit]
Description=qBittorrent Daemon Service
After=network.target
[Service]
User=pi
ExecStart=/usr/bin/qbittorrent-nox
ExecStop=/usr/bin/killall -w qbittorrent-nox
[Install]
WantedBy=multi-user.target 
Lastly, enable the script.

sudo systemctl enable qbittorrent
3. pyLoad

The final part of the exercise was to set up a download manager. I had briefly given thought to JDownloader but decided against running JRE just for it. Hence, I opted for pyLoad instead. The tutorial listed over here works fine for the most part but needed quite some tweaks along the way. For the sake of completion, I will list all the steps in brief.

1. Create system user for pyload

sudo adduser --system pyload
2. Edit /etc/apt/sources.list to be able to install the dependencies. For Raspbian Stretch, the source URLs are as follows:

deb http://mirrordirector.raspbian.org/raspbian/ stretch main contrib non-free rpi
deb-src http://archive.raspbian.org/raspbian/ stretch main contrib non-free rpi
3. Update package list and install dependencies.

sudo apt-get update
sudo apt-get -y install git liblept4 python python-crypto python-pycurl python-imaging tesseract-ocr zip unzip python-openssl libmozjs-24-bin
sudo apt-get -y build-dep rar unrar-nonfree
sudo apt-get source -b unrar-nonfree
sudo dpkg -i unrar_*_armhf.deb
sudo rm -rf unrar-*
4.  Create symlink to get "spidermonkey" working.
cd /usr/bin
sudo ln -s js24 js
5. Unlike in the linked tutorial, I had to first switch to the stable branch before I could initiate the pyLoad setup. This is done as follows:

cd /opt
sudo git clone https://github.com/pyload/pyload.git
cd pyload
git branch -r
git checkout stable 
 6. The next step, as per the linked tutorial, should have been the initiation of the pyLoad setup using:


sudo -u pyload python pyLoadCore.py
However, doing so produced the following error: "ImportError: No module named pycurl". Hence the next logical step was to install pycurl:


sudo pip -v install pycurl --upgrade
This in turn resulted in the error: "InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-IsWfyN/pycurl/". This was resolved by:


sudo apt-get install libcurl4-gnutls-devpip
 As you can now guess, this in turn resulted in yet another error: "Failed building wheel for pycurl" which was remedied as follows:


sudo apt-get install libgnutls28-dev

After all this effort, I was finally able to install pycurl using the command mentioned previously...


sudo pip -v install pycurl --upgrade 
...and execute the pyLoad setup using a slightly shorter command:


 python pyLoadCore.py
On the June 2018 release, I was able to get pyload wizard started by executing
sudo -u pyload python pyLoadCore.py 

The final tweak was to get pyLoad running on boot with the commands executed in a manner similar to what has been already covered for qBittorrent.

sudo nano /etc/systemd/system/pyload.service 
[Unit]
Description=Python Downloader
After=network.target
[Service]
User=pyload
ExecStart=/usr/bin/python /opt/pyload/pyLoadCore.py
[Install]
WantedBy=multi-user.target
 sudo systemctl enable pyload.service
Beyond Home

The benefits of this setup can be extended beyond the home network, though a lot depends on the vagaries of the network setup.

Pi-hole can be put to use on external networks by accessing the home network over OpenVPN, though speed and latency might be factors to consider. It can also be setup as a Public DNS but it is extremely risky and not at all recommended.

qBittorrent and pyLoad can be simply accessed using the IP address and port, provided they have been setup to be accessible from outside the LAN. For dynamic IP addresses, the Dynamic DNS (DDNS) option available on most routers can be put to use and my suggestion would be to pick up a cheap $1 domain from NameCheap for this purpose. However, if you happen to be under a multi-layered NAT network under ISP control like me, then there is no option other than to pay for a static IP for public access.

Thankfully, there is a last resort to access the Pi over a public network. The licensed copy of RealVNC that comes with Raspbian offers Cloud Connect that enables one to remotely control the OS and thereby all applications on it. It is quite cumbersome to use if your intent is to only load some links on qBittorrent or pyLoad, but it is better than nothing.

Thus, the Pi can be extremely useful even when used in a rather sedentary capacity and you grow to appreciate the efforts that everyone has put in to make this possible.

Musing #57: Steam Link on Fire TV


The release (or lack of it) of the Steam Link app caused a lot of brouhaha in the past month. While it it is meant for mobile devices, it undeniably adds a lot of value to the Fire TV and for that matter to all Android devices. It is a must-have that would have certainly made it to my list of  'The Essentials' were it available back then. It is not officially available on Amazon, so your best bet is to sideload it.

As I mentioned previously in my review of the AFTV3, the Ethernet adapter doesn't make a whole lot of sense as it is limited to 100 Mbps. However, it would be more than enough in this case as Steam Link requires a maximum of 30 Mbps for streaming. Unfortunately, I had to rely on the 5 GHz WiFi network (Steam Link doesn't support 2.4 GHz) with the TV being 25 metres away from the router, separated by a wall. This issue is compounded by the fact the 5 GHz receiver on the AFTV3 is exceptionally weak.

After playing with the settings, the only way I could get Steam Link working on the AFTV3 over such a long distance was by switching the 5 GHz channel bandwidth to 20 MHz. This significantly reduces the throughput but is a necessity for my current setup which I hope to change soon. Over the 20 MHz channel and at a distance of 25 metres, Steam Link works unimpeded in the 'Balanced' mode which uses 15 Mbps. I was even able to get the 'Beautiful' mode, requiring 30 Mbps, to work over the 20 MHz channel but it was inconsistent. On the other hand, it worked exceptionally well over the 40 MHz channel as can be seen below, but the AFTV3 was unable to sustain the signal over the distance, resulting in frequent disconnections. Nonetheless, this is an issue that can be easily resolved through some rearrangement.


Steam makes it quite evident that the software is in beta and that AFTV is not officially tested.


 However, as long as the network is up to it, the AFTV is more than capable of streaming.


Inability of the network to stream properly is indicated with the frame loss and network variance.


Setting up Steam Link is extremely easy as it essentially requires pairing the TV with the host PC using a PIN.


Additionally, the Steam Client on PC requested the installation of additional audio drivers once the setup was done, but I presume this might depend on the setup. I had sold my Xiaomi Bluetooth controller a few months back so I didn't have a controller to pair with Steam. However, I did have my Apple Wireless Keyboard and Logitech M557 paired to AFTV which ought to have done the job. 


While the keyboard worked fine with the Big Picture mode, v1.1.3 of Steam Link that I installed initially didn't support the mouse which was subsequently rectified in v1.1.4, indicating that Valve is actively paying attention to user feedback. At present, the lag isn't too bad, but the mouse controls are too sensitive which I presume is due to the fact that the tuning has been done as per analog controllers. It might make sense to pick up the Xbox One S or Steam controller for universal compatibility.

With the initial impression being quite good, one can only hope for Steam Link to work seamlessly once it comes out of beta. Perhaps the Steam Sale will become a lot more attractive for AFTV owners.

Review #51: Amazon Fire TV (Gen 3 - 4K HDR) ★★★★☆


Normally, whenever I get a new device, the instinct is to analyse it in depth. Unfortunately, certain constraints prevent me from doing so with the Fire TV 4K, primary of them being that I have no 4K display devices at present. However, a lot of thought had gone in to purchasing this device for a 1080p non-HDR TV, even though it is not officially available in India. So, I would like to share these thoughts along with the experience of setting it up so as to get the most out of it. Thus, this article will straddle the line between a tutorial and a review, but I have decided to classify it as the latter since this article, while being instructional, is still appraising the product.