Nothing's wrong with you man. I get irritable for no damn reason too when i dont work out. I'm not bipolar. Some days are just rough and in those times - what you do, and especially what you don't do - are what makes the next day better or worse. it will definitely pass.

I'm the same way.. If you cannot make it to the gym at that point in time find a room all to yourself with no one else.. your car what ever.. Close your eyes breath in through your nose and out through your mouth slowly for as long as it takes.. Don't think about what's pissing you off just think about breathing because this moment alone is about you and nothing else.. You may find after this your brain begins to approach things that were pissing you off a little differently for the time being..
Every one has their little fix for the day whether it's a cigarette, a beer, a joint, binge eating, a video game, or a movie to decompress.. Craving a workout just seems so much better so I don't think you should be looking at this like a crutch but rather view it as your better off than most people.. Just try to work out in the morning and if you have exhausted all of your body parts just do a little cardio or a bike ride