You were right about the top part as well. Once I removed the lower panel, there were two screws exposed. After removing them, I was able to pull the whole tray and lid out.
Anyway, long story short, I got everything apart and applied plasti dip to them, but unfortunately with those horizontal grooves in the panel, it looked horrible. I peeled it and put it all back together. I guess I'll have to find a replacement, I thought I saw a thread where someone had used the dash out of an R/T or something....