I currently use the AUX jack to listen to my iPod. I ordered a small cable online and try to gently close the armrest so that I can read the iPod and change it. Not too long after I started using it I started having audio problems. I find if I hold the iPod in a certain location, sound will not come out of the driver or passenger speakers or both and rotating the cable while it's in the ipod or moving the position of the iPod changes how it works. This is a problem because once it works if I even touch the iPod the sound will not work right. It is so frustrating! Has this happened to anyone? I imagine it is because I close the armrest on the cord but I am not sure what else to do. It is really tiresome to open the armrest every single time I want to change the song (which is frequent). It might have seemed convenient when they made it but I really wish the jack had been in the dashboard area.
Sounds like you either need a new cable (perhaps a longer one). The armrest cover is not normally a tight fit, so it shouldn't be causing a problem. Also, you may have an intermittant audio jack on your iPod. I have a XM tuner (not the Prius kit) mounter in the flip down panel below the car's stereo. I built a little switch box that I have mounted nect to it with a stereo jack on the front. This feeds the Aux input with either thee XM or an iPod (or a Sansa Fuse in my case). The I ran an audio cable from this switchbox down under the center console and into the Aux jack there (I had to drill a hole in the bottom to run the wire thru). .. This way I didn't have to mess around with the wiring to the radio. The added audio cable length (from switch box to Aux jack and then back to the JBL) does not seem to roll off the highs. Note: I used Velcro to attach the switch box as well as the XM tuner. That allowed for easy installation, and removal.
Get a higher quality cable. Chances are the cable right next to the jack has broken, and is giving you an intermittent connection in one/both channel.