In order to change the default settings in Mac OS X (for example, duplex printing), it is recommended to use the CUPS web interface (since the changes seem to be impossible to save as a default via the normal print settings dialog).
You should access
http://localhost:631, then the menus "Administration", "Manage Printers", then select your printer. From the pull-down list named "Administration", select "Set Default Options" and make the desired changes.
For instance, in the tab "General" set the Duplex option to"DuplexNoTumble", which is equivalent to Long-Side Binding setting in the normal print settings dialog.
UPDATE #1: On Mac OS X 10.9 the name for the duplex options has changed,
and is now "Print Style" with the value "2-sided printing"
UPDATE #2: If you get as access error, you may need to enable the web interface by running the command:
> sudo cupsctl WebInterface=yes