How do ensure that your application properly handles errors, especially when relying on third parties, such as payment processors? Is it easy to verify that the right things happen when the wrong things happen? Last week’s article Strategies for Rails Logging and Error Handling discussed some techniques to setup a good error handling strategy. Here’s some techniques to verify that your application does what you expect it to do when things go wrong. The key message is to check how your application handles errors, before your customers do.
Table of Contents
Your Code Depends on Outside Systems (That Might Raise Errors)
Suppose you’ve created the super-duper Rails storefront application that takes online payments. You may even have some unit tests that verify the code. Then you get the dreaded call that customers are being charged twice and their orders are not processed. WTF?
It’s not entirely obvious how to verify proper error handling when outside systems fail, or even when odd errors are raised from your own code. Payment processing deserves some special attention because it’s a dependency on an outside service (the payment processor) and will typically require database updates based on the result of the payment processing. If you’re updating several tables, then you’ll want to use a transaction to ensure that all or nothing saves. While code review and manual testing are good first steps, you should consider a few extra steps with error handling for sensitive parts of your application.
Verification of Error Handling Strategy
Typically, error handling code is not well tested. It’s much more common to test the “happy path” of everything going right.
Let’s look at hypothetical example and some tests that can flush out some errors.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | |
So what can go wrong?
Payment Processing is Like a 2-Phase Commit
Conceptually, you want a transaction, such that it’s all or nothing. If the charge goes through, then so does everything else. Payment processing like a 2-phase commit, except one has to handle all the what-ifs to be sure that it’s handled correctly.
The general steps of payment processing are like this:
- Connect to outside resource to make charge.
- Update database records indicating charge successful.
- Fulfill the order.
Rails transactions work such than any exception in the block will cause the transaction to be rolled back. The problem with the above code is what happens if fulfill_order throws an exception? The customer has been charged, the order was updated to reflect payment, but then ka-boom and an exception is raised, and any database updates to the order are rolled back, but the payment is not refunded. The customer is confused as there is a charge but nothing else. How could you have tested (and avoided) this?
Brute Force Methodology
You can simulate error conditions by manually placing =raise “any error message”= statements in your code, and then testing, say in the UI manually. This is a good first step to verify that your error handling is working correctly. You might raise a specific error, if say your payment processor throws a specific type of error.
For the above example, the different methods referenced, such as process_order can get modified with a single line at the beginning, which would be:
1 2 3 4 | |
Then go into the UI and test placing an order. Consider the following questions:
- Was the right error message displayed to the user?
- Was the right information logged at the correct log level?
- Was an automatic email sent regarding the error?
See my prior article Saner Rails Logging for the answers to #2 and #3.
By applying this technique to each of the components of completing a purchase, one can flush out (and handle) nearly all of the different possible errors that could affect a purchase. Give this technique a try in some critical section of the code. You’ll be surprised how well it works. Before giving you the fix to the above code, let’s see if we can write unit and feature tests on our error handling.
RSpec Unit Testing of Errors
It turns out that with stubbing in rspec, it’s easy to test error handling! RSpec provides a nice mocking library. The test code would look something like this. Pay attention to the call to stub.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | |
This test code ensures that the error handling of purchase_cart will catch an error from fulfill_order, and properly refund the payment and rollback any changes to the order record.
Here’s an improved version of the Order#payment_method above:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class Order
def purchase_cart
error_message = nil
begin
Order.transaction do
# user has a credit card info, returns either error_message if failed or charge_details if success
error_message, charge_details = PaymentGateway.charge user, total
set_charge_fields_and_save user, charge_details unless error_message # update the order to indicated purchased
end
fulfill_order # do lots of complicated stuff to process the order, do this outside of the original tx, so that the payment info can be committed.
rescue => e
Utility.log_exception e # Unified strategy for error handling including email notification, see below
refund_charge if charge_details # If there's an error here, then sys admins will have to manually refund the charge.
throw e
end
error_message # return any error message if there is one
end
end
- There’s a block to catch the exception which is separate from the transaction block. The rescue properly handles the case of an a charge being made and needing to be refunded. Utility.log_exception will ensure that all the right things happen with this sort of error (see code for Utlity.logException).
- fulfill_order is moved outside of the transaction block. This allows the transaction to complete, and then the order_fulfillment takes place. If there’s an issue in fulfilling the order, that can be dealt with separately from the original charge. In other words, the customer can successfully pay for the order, and the store can deal with the failure to fulfill the order.
RSpec Capybara Feature (Integration) Tests of UI Errors
It’s possibly more important and sometimes easier to do the verification at the integration level in RSpec feature specs using Capybara with PhantomJs and Poltergeist. The secret sauce is the same use of the same stubbing technique as above to replace some key methods such that they throw an exception. This sort of technique works amazingly well to ensure that application will do the right then when an unexpected failure occurs, from the logging and emailing of the error message to the browser display to then end user.
I tend to develop such a test in an iterative manner:
- Make sure you’ve got tests on the “happy” case where the story goes as planned.
- Then introduce test cases where have bits of code like this that will raise an error at an opportune time.
1
| |
- Allow the test cases to fail, and put in screen shots (in Capybara with phantomjs, that looks like this:
1
| |
Setup this method render_page in a spec helper file like this:
1
2
3
4
def render_page name
path = File.join Rails.application.config.integration_test_render_dir, "#{name}.png"
page.driver.render(path)
end
- Put in some assertions that the page shows the correct error and the records in the database have the right values.
- You can even
Here’s an example that tests a failure of the Stripe payment API, including verification that an email was sent signifying an error:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | |
Conclusion
If you aren’t simulating how your application responds to errors, then you’ll eventually find out, and the result might not be as good as you’d prefer. You can simulate errors with the very simple and quick technique of a well placed =raise “some error”=, and then testing in a UI. Or you might prefer the robustness of unit or feature tests using stubbing. Either way, the key message is to check how your application handles errors, before your customers do.
Related Post: Strategies for Rails Logging and Error Handling
start guard LiveReload
