To completely remove Microsoft Remote Desktop from OSX you need to perform the following
Bin the MRD app
Drag the Microsoft Remote Desktop app to the bin.
In Finder, select Go > 'Go to Folder', and enter the following path:
/private/var/db/
Locate the RemoteManagement folder and bin it.
Back to Finder, again select Go > 'Go to Folder', and enter the following:
/Library/Containers/
Following you entering the above and nothing happens (as did for me), you need to enter the full path
/Users/{name}/Library/Containers/
for me this was
/Users/andrewfletcher/Library/Containers/
Locate the com.apple.RemoteDesktop folder and bin it.
Back to Finder, select Go > 'Go to Folder', and enter the following:
/Users/{name}/Library/Application Support/Apple/Remote Desktop/
Locate the Client, Shared Settings, and Task Server directories and bin them.
Back to Finder, select Go > 'Go to Folder'. Then enter the following path:
/Users/{name}/Library/Preferences/
Locate the com.apple.RemoteDesktop.plist file and drag it to the bin.
Close any instances of the Remote Desktop widget, if installed.
Back to Finder, select Go > 'Go to Folder' and enter the following:
/Users/{name}/Library/Widgets/
Locate the Remote Desktop.wdgt file (if installed) and bin it.
Delete the contents of your bin
The file that causing me issues was
/Users/andrewfletcher/Library/Containers/com.microsoft.rdc.macos/Data/Library/Preferences/com.microsoft.rdc.macos.plist
So I deleted the com.microsoft.rdc.macos directory
In the directory /Users/andrewfletcher/Library/Containers/, I ran the following command
rm -rf com.microsoft.rdc.macos