The typical EM agent upgrade leaves the old Oracle Homes in place on your hosts. That includes the old home for the agent binaries and also the homes for each upgraded plug-in.
You can clean all of them up through the same Agent Upgrade Console that you used to push new agent out. Select the same Upgrade Agents from the drop-down.
Select the Post Agent Upgrade Tasks tab to expose the Clean-up Agents functionality
Search for the Agents available for Clean-up then make your choices. Of course, you can filter the search as you see fit.
After the selections are highlighted, hit the Submit button to create the EM Job.
Like all EM Jobs you can click through to watch the progress and to check the run-time logs (and you should)