Here's a note in case I forget what the actual reason for this message was or how it was resolved.
After not using Google Music Manager for a while, I tried to log in to upload a couple of recordings to share. Unfortunately, I could not log in on the PC. It was giving me the "Could not identify your computer" error message after going through the 2-step authentication.
Reading online pointed out that, for whatever reason, Music Manager clings to a device's MAC address. This was a good pointer.
Checking the network device properties / configure / advanced, there was only one property available - Network Address. Fortunately, this is exactly the one I needed. It was set to Not Present, as it normally should. As I was connecting using USB tethering, I guess this value was never really set.
So I set the value manually to 1F-2E-3D-4C-5B-6A. This is a simple value that follows the specs. And, voila! It worked.
In the meantime, I also tried switching to a Wi-Fi connection but that did not work somehow. In the properties, there was no property for Network Address so I could not see what it was nor modify it.