If the order has a status of New, you can click the red 'X' to the right of the purchase order to cancel.

If you are within the Purchase order, you can select Cancel under the actions drop-down.