Introduction. #
So i now have a small flexible job at studentaanhuis (student at home) and its basically helping other people who are totally non-tech with some issues/problems they are facing. It can be anything from networks to computers to software, just that we dont provide computer hardware services thus no repairs or upgrades or cleaning (unfortunately), but i like the more software side too!
Day 1. #
The problem/issues. #
SO i got a new appointment in my mail and i could see it in my work portal. It was about networking problems. The customer had bad WiFi in some places in their house. They had 5x APs but the customer thought some where broken due to them not lighting up a blue light and also cause WiFi in the garden sucked and they had a AP in the garden. And their TV did not want to connect to network and thus they could not open Netflix.
So i went to their house. #
SO i saw that the address was pretty close by. 12 Minutes on the bicycle is all it took to get there. When i got there, it was a nice big house and i expected them to be pretty rich people as it all looked good and they had some nice looking cars in front of the house. The appointment was planned at 10:00 but i was there 2 minutes before 10 as i calculated some extra travel delays in. Then a woman opened the door and then i also saw the man who contacted the organization (studentaanhuis) i am with. It seemed to be a couple that was not really elderly. They also seemed to have kids but they where not there when i came.
Reception. #
So they asked me if i wanted to drink something and then i asked them what the issue was and the man showed me a bit around the house and explained the problems and he shown the System which seemed to be a Ubiquity Unifi system with 4x Unifi AP AC Pro access points and 1x Unifi AP AC Mesh which hung under the canopy. And he showed me the APs and indeed, 2 of them did not seem to have a light or it was turned off. And the TV had indeed no connection. It was 2 access points on the 2nd floor, one in their bedroom and the other on the other side of the 2nd floor. Then on the main story, there was one just about above the door and one in the room with the TV and the final one hung under the canopy which was the mesh one.
Initial debug. #
So then i started looking throughout the house and seeing what i can spot. I then went to the man and asked if he had the Unifi app on his phone. He did not and i asked him if he could download that. After signing up for an account/creating one, we could see all 5 AP’s popup in the app and what seemed to be a Unifi switch. We also saw that it was externally managed/managed by someone else. Pretty cool to see a Unifi system as i never have worked with one myself. I expected some cheap crappy WiFi extenders or the ISP provided ones, but nope, a full expensive Unifi system. Ubiquity can be considered the Apple of networking gear and is very high end and is used by millions of businesses.
Who installed it? #
After asking them about it, they told me that a few years back someone they knew/a friend had installed the system for them. Also as they told me that it was all complete magic to them and that they did not know anything about it. Then as last i asked them if i could see their fuse box which is often where the router and other stuff is located. It was actually outside the house next to the front door and had a lock for which they had the key.
What a mess! #
They opened it up and what i saw there was a complete mess! It was a whole spaghetti of cables and there where some long 10m cables used for distances of 1 Meter or even less and some cables where not even rolled up and everything hung by the power cables… Trying to not mind the mess, i did see how it was connected. I saw 2 switches, a Unifi managed smart switch with 4x POE ports and 4 non-POE and a TPLink POE managed switch, again with 4x POE and 4x non-POE, both switches had 8 ports. I also saw that they had a Ziggo modem which provided the internet and it was hooked up via Coaxial. I have the same router at home. No fiber but a Gigabit connection is also not bad (also considering its COAX). They also had the older white Ziggo modem (with the awful intel puma chip-set) but i told them that they could toss it and also it was not hooked up anyways.
I knew the AP’s worked over POE and thus hence 2 POE switches where installed. But funnily enough it seems like all 4 APs where connected to the Unifi switch and thus occupied all POE ports but somehow the TPLink POE switch had only 1 Access point hooked up and thus was basically serving like a POE injector rather than a switch. And also everything was quite dusty/grimy. All cables where cat5e rated which s fine for 1Gig and also the customer did not do any heavy bandwidth stuff anyways.
Hello Raspberry-PI! #
I also did see a black box and after recognizing the Ethernet jack and next to it 4 USB ports and a power cable on the side and an sdcard in the back, i knew it was some single board computer. Opening up the black box, it was a raspberry pi 3B+! I did not yet know which task it had but i later discovered that it was actually supposed to be the Unifi controller. Basically in Unifi setups, you have 1 central controller managing the whole network and it basically being a director of the whole network. It had a 16GB transcend sdcard and was powered by a power adapter and connected via a cat5e cable to the Unifi switch on port2. Seems like that friend who installed it for them also was into raspberry pi and Linux which is funny!
Starting with my work. #
So i decided to start with the issues and trying to solve them. In the App we saw the whole system was managed by that friend/person they knew who installed it for them. We tried some login combinations but no luck. Then i told them that the best way to be able to manage the system and possibly solve issues along the way was to reset all the APs and then basically give the customer control over the system so if any errors occur we are not dependent on someone outside. And then we easily could manage and re-add all APs and reset all settings to factory defaults. I decided to start with that mesh AP in the canopy. Because it hung a bit high, they provided me with a ladder. Then i had to wiggle it out of the mounting bracket and then i could access it. Then i opened up a small cover which was just clipped on and then i found a small reset pin-hole. They gave me a needle i could use to reset it. I decided to reset this one first so we could see what it does. After letting it reset itself and clear all configurations in its config partition and then reboot, we could see it in the App and it was ready for adoption in a Unifi system. When we had it show up in the app and added it, it did complain about the fact no Unifi controller could be found, but it did allow setting it up without one, but i instead waited on setting it up.
I did try to reboot the PI but it seemed to not show up in the network as a Unifi controller and after looking up the IP in the modem’s device list and then accessing the WebUI i got some errors so then i told the customer that i would have to reset that raspberry pi as some stuff was broken/not working. I told them that i would have to reinstall the software on the sdcard and then reconfigure it. It was also already quite late and i had other customers so i signed it off there and spoke with them about coming back the next day (15 July) and that was no problem and they where already grateful to see what i knew about the system and where excited for me to come back the following day and install and setup the whole thing. Also i did not have my laptop or other stuff with me so i told them that the next day i would bring my laptop and a sdcard and sdcard reader (built into laptop) with me. We planned the appointment the next day at 16:00.
Day 2. #
So it was the next day (15th July) and after working off some other appointments i first went home (as i was done in a few hours). Then at 15:45 i took the bicycle and cycled to their house again. When i came there, their kids where also there and some other people i did not know and also did not really give any attention to. They where all interested in seeing me mess with it and they all where like “Its nice to see that someone is here to fix our WiFi. Cant wait till we have good WiFi again.”. I started with the raspberry pi. I had brought my little latitude 7290 laptop with me which had Debian Linux on it. It also had a micro sdcard reader built in. I also took with me an extra 32GB sdcard just in case.
First lets look at the sdcard that was already installed. #
SO i pulled power from the PI and then took out that 16GB transcend sdcard and plugged it into my laptop. Then i could see the typical raspberry pi stuff like a boot partition labeled “BOOT” holding the Linux kernel and other startup files it needs and then a 2nd partition called “ROOTFS” which was an Ext4 file-system containing the typical basic Linux file-system stuff and directories. And because my laptop was running debian i could easily see the files (Windows dislikes ext4 file-systems or any other FS that is not the typical FAt or ExFat or NTFS). Then i quickly found the Unifi map in /var/lib/ and thus that confirmed my suspicion that the PI was the central Unifi controller. I also found some .unf backup files which are automated backups created by the Unifi controller software on the PI.
Sacrificing my own sdcard. #
So because it did have some backups on it and i did not want to do any irreparable harm, i decided to sacrifice my own sdcard i had brought with me just in case. This way i could install a fresh copy of Raspbian and then install the Unifi controller software atop of that. SO i installed Raspbian using the Pi Imager software. Then in the software i set a username and password and also i enabled ssh as i did not have a screen or keyboard with me and its a headless setup anyways. After letting it boot, i found it in the routers list and got the IP.
This will take a looong time! #
Then i ssh’ed into the pi and ran some system updates/upgrades which took a loooooooooooooooooong time! I think i was easily already wasting 1 hour on just updates… Crazy! But it also was very hot that day so the PI must have been thermal throttling (where the CPU will slow down so it does not fry itself). After that was done, i installed the Unifi software using those prebuilt install scripts. The script also took its sweet time and annoyingly, it also ran some system updates/upgrades causing even longer time. I think i was easily wasting 1,5 hours just on the software install part.
And then also letting the script setup and configure everything which took another 30 Minutes…
De-spaghettifying the cable mess. #
So i decided to in the meantime try to clean up the cable mess (of course without disturbing the PI). I started by disconnecting all AP’s from the Unifi switch. Then i also disconnected the TPLink switch and some other unnecessary cables. Of course being careful to not accidentally pull the power or Ethernet from the PI which could mess up stuff/halt progress. I disconnected that long 10M cable and saw a shorter nicer rolled up Ethernet cable which was laying inside which i simply replaced the annoyingly long cable with. After tidying some things up, i got it to a state where it was less messy and i could easily see how everything was connected.
After around 2,5 hours the PI was done with installing stuff (ended up taking so long as i had to fix some issues). I also in the mean time that the PI was installing software, i decided to distribute the APs across the 2 switches. So 3 AP’s on the Unifi switch and 2 on the TPLink. And i routed all cables a bit nicer. What was nice is that 3 out of the 5 APs had a label on their cable so i knew what APs those where and what the cables where for.
I literally ripped out and disconnected everything and then reconnected everything more neatly. I also found a charger that was not connected to anything and remove that old white modem and remove 2 long cables which was nice. And i dusted off the switches and modem and the PI.
Finishing stuff. #
So after it was done installing the software, i could access the WebUI via my laptop and the pi’s IP address with port 8440. I could then start setting up the this new Unifi network. I walked to them (they where in the garden speaking with other people) and then asked him to help me set it up. We logged into their account and entered the stuff in the Unifi controller’s WebUI and in a few minutes it was setup. Then because we had reset one of the 4 Unifi AC Pro AP’s and the mesh one in the canopy already the day before, they already where showing up and ready to be adopted into the Unifi system. SO we added them first. When i added them, they also automatically updated and then started configuring themselves. After those updated and where setup, i asked the customer to test his connection in the garden, which finally worked nicely! Then i started resetting the rest of the AP’s and adopt them in the Unifi system and of course they all updated and set themselves up and did their things.
Then we tested the network and it worked fantastically! The switch was still managed by the old managing account but it functions perfectly fine as a dumb switch and we dont need to do any fancy stuff anyways so it works fine and i did not bother resetting that. We also cleaned off/dusted off some of the APs which where kind of dirty.
After that i tried connecting the TV to the network. It did not work at first but after pulling the plug of the TV and plugging it back in, it worked! And Netflix opened successfully. Classic IT support joke/method that somehow works most of the time: “Just unplug it and plug it back in”.
Signing off. #
So after i was done (it was a hell of a task but i enjoyed every minute of it) it was already 19:15 so quite late already! I also underestimated the amount of time i would spend on it. I Though: “Oh well i could finish it in 2 hours” but it ended up taking near 3.5 hours. Also i worked past dinner time (as we planned to eat around 18:45) but my mom and dad saved some food for me.
I spent in total 5 hours on it but it was worth it. Also i enjoyed working on the system as i never touched/worked on a Unifi system and they really are the Apple of network devices (premium, cost a fortune and are genuinely just nice looking and high end).
The people where very nice and where really happy that everything worked again! I dont know what it is, but i had some of the best/nicest experiences with the sweetest people who all live in those big houses and are probably very rich too. I expected them to be a bit more stingy/strict but not at all! They asked me multiple times if i wanted something to drink (even after my glass was empty after i already had something) and they sometimes had sweets/cookies i could take. And they made me feel at home and in general i felt comfortable. And its nice to see how trusting some people are to let you take care of something without them staring at you the whole time. They let me take care of it all and even asked “Could we help? If not, do your thing but dont hesitate to ask us for help. We are outside in the garden so you know where to find us”.
In the end they even gave me 50 Euros as a tip for the work, crazy!!!! And he promised to leave me a good review on studentaanhuis. They really where grateful and this is the 2nd time i have been tipped! Previously i had been tipped 20 Euros which was also nice and was from very nice kind but rich people. And also i like that they fully believed in the “next day, everything is fixed” thing and i am just a very hard working guy who does not give up easily and enjoys working on stuff for hours.