The program recordmydesktop saves the videos in OGV format. To be able to embed this in a web page with the HTML5 video element it needs to be either in OGV or MP4. And not every browser support every format (event if specified in the specs). So it is best to have both formats available in the web page.
<video width="1024" height="656" controls > <source src="videos/first-steps.mp4" type="video/mp4"> <source src="videos/first-steps.ogv" type="video/ogg"> Your browser does not support the video tag. You can download the <a href="videos/first-steps.mp4">video</a> for offline viewing. </video>
To get an MP4 file from the OGV file you can use mplayer/mencoder for it.
mencoder first-steps.ogv -lavcopts vcodec=libx264 -o first-steps.mp4