Testing PATCH Requests

Like PUT requests, PATCH requests modify an existing record.  But PATCH requests are much tricker to test!  This is because a PUT request modifies an entire record, whereas a PATCH request modifies only one part of the record.  There are many different operations that you can do within a PATCH request: you can add, replace, remove, copy, and move a value in your record.  I'll describe some examples of each and discuss the various ways you can test them.  

Let's use a very simple example of a data record for patching:

The JSON description of the first record would look like this:

{

    "id": "1",

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551000",

    "workPhone": null

}

Notice that our first record does not have a work phone.  Let's do an add operation with our PATCH request in order to add one:

The URL for the request would look something like: https://app/customer/1, where the "1" is representing the id of the customer that you want to patch, and of course the http verb used should be PATCH.  The body of the request would look like this:

[

    {"op":"add","path":"/workPhone","value":"8005551001"}

]

The "op" in this case is referring to the operation that you want to do for your PATCH.  Here we are using an add operation.  The "path" is showing which field you would like to add a value to, in this case the work phone field; the "value" is value that you want to add to that path, in this case the phone number itself.  

When this operation is completed, the JSON description will look like this:

{

    "id": 1,

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551000",

    "workPhone": "8005551001"

}

Here are some ways to test a PATCH add operation:

• Happy Path- patching where there is currently a null value
• patching over the null value with an empty value of ""- this should add an empty string
• patching over an empty value of ""- this should replace the existing empty string
• patching over an existing value- this will replace the existing value, but ideally the replace operation should be used here instead
• adding a value with too many or too fewer characters than allowed- an appropriate error message should be returned
• adding a value with characters that are not allowed- an appropriate error message should be returned
• adding a value of the wrong type, such as adding an integer when a string is expected- an appropriate error message should be returned

Next, let's take a look at the replace operation.  Let's use the second record this time, and we'll replace the home phone.  This is what the record currently looks like:

{

    "id": 2,

    "firstName": "Amy",

    "lastName": "Jones",

    "homePhone": "8005551002",

    "workPhone": "8005551003"

}

The URL for the request will be https://app/customer/2, because we are patching the second record, and the body of the request will be:

[

    {"op":"replace","path":"/homePhone","value":"8005551111"}

]

This operation will replace the original home phone of 8005551002 with a phone number of 8005551111, so the record will now look like this:

{

    "id": 2,

    "firstName": "Amy",

    "lastName": "Jones",

    "homePhone": "8005551111",

    "workPhone": "8005551003"

}

To test a PATCH replace operation, here are some things that you can try:

• Happy Path- replacing one value with another
• replacing the value with null- a remove operation would be better for this, but this should still work
• replacing the value with the empty string ""- this should work
• replacing a null value with a value- this will probably work, but it would be better to use the add operation
• replacing with a value with too many or too fewer characters than allowed- an appropriate error message should be returned
• replacing with a value with characters that are not allowed- an appropriate error message should be returned
• replacing with a value of the wrong type, such as adding an integer when a string is expected- an appropriate error message should be returned
• replacing where the existing value is bad in some way, such as having too many characters or having the wrong format- in this case the new good value should be allowed to replace the bad old value

Now let's look at the remove operation.  We'll start with this record:

{

    "id": 1,

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551000",

    "workPhone": "8005551001"

}

To remove the home phone of this customer, we would again use this URL: app/customer/1, and we would use this body:

[

    {"op":"remove", "path":"/homePhone"}

]

We will get this as a result:

{

    "id": "1",

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": null,

    "workPhone": "8005551001"

}

A remove operation is fairly easy to test.  We just need to verify that the value has indeed been removed and replaced with null.  It's also good to verify that no other value was removed by mistake, such as removing both phone numbers instead of just the home phone. We can also test that it's not possible to remove a required field- this should return an appropriate error message.  

Now let's look at the move operation.  Looking at the previous state of John's record, let's imagine that we have it wrong, and that what we have for his work phone is actually his home phone.  To move the phone number, we would use this request body:

[

    {"op":"move","from":"/workPhone","path":"/homePhone"}

]

In this example, the "from" indicates where the current value is, and the "path" indicates where you would like the value to move to.  After this operation, our record will look like this:

{

    "id": "1",

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551001",

    "workPhone": null

}

Here are some ideas for testing a move operation:

• Happy Path- moving an existing value to a location where the value is currently null
• moving an existing value from location A to location B, where there is already a value in location B- in this case, the value in location B will be replaced with the value that was in location A
• doing a move from A to B where the value of A is null- this should return an appropriate error message, because there's nothing to move
• doing a move from A to B where the validation constraints are different in location B, so A's value should not be allowed- this should return an appropriate error message
• doing a move from A to B where the value in A is bad- this should return an appropriate error message
• moving a value from one location to a location that does not exist: this should return an appropriate error message

Finally, we'll look at the copy operation.  This will copy an existing value and put it in a different location.  Let's take this example:

{

    "id": "1",

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551001",

    "workPhone": null

}

If we want to copy the home phone so that the work phone will be the same number, we can do this request:

[

    {"op":"copy","from":"/homePhone","path":"/workPhone"}

]

The "from" field shows us the location of the value we want to copy, and the "path" field shows us the location we'd like to copy to.  Our record should now look like this:

{

    "id": "1",

    "firstName": "John",

    "lastName": "Smith",

    "homePhone": "8005551001",

    "workPhone": "8005551001"

}

Testing a copy operation is very similar to testing a move operation:

• Happy Path: copying a value from one location to another where the value is currently null
• copying a value from location A to location B, where location B currently has a value; this value will be replaced by the value in location A
• copying from location A where the value of A is null- this should return an error message
• copying from location A to a location that does not exist- this should return an error message
• copying from A to B where the validation constraints are different in location B- this should return an error message
• copying a bad value from location A to location B- this should return an error message

An important thing to note when patching is that PATCH requests can be chained together.  For example, if we start with this scenario:

{

    "id": 2,

    "firstName": "Amy",

    "lastName": "Jones",

    "homePhone": "8005551002",

    "workPhone": "8005551003"

}

And we run this PATCH request:

[

    {"op":"remove","path":"/homePhone"},

    {"op":"replace","path":"/workPhone","value":"8005551111"}

]

We will wind up with this result:

{

    "id": 2,

    "firstName": "Amy",

    "lastName": "Jones",

    "homePhone": null,

    "workPhone": "8005551111"

}

When we chain together requests in a PATCH, we need to test that any invalid operation will result in the entire request being invalid.  For example, if we had instead tried this request:

[

    {"op":"remove","path":"/homePhone"},

    {"op":"replace","path":"/workPhone","value":"NOTAPHONENUMBER"}

]

we should receive an error message, and the first part of the PATCH should NOT have been executed.  In other words, we should still have a value in for the home phone.  

As you can see, PATCH requests are almost a set of HTTP verbs in themselves!  They should be used very carefully in an application, because there are so many different ways they can go wrong.  If your application is using them, be sure to test them alone and in combination.  Be sure to test both the Happy Path and the many ways they can fail validation.  If you are able to put bad data into your database, be sure to test patching over bad data.  

In next week's post, I'll discuss the DELETE request, and also how to create a Postman collection!  

Testing PUT Requests

In last week's blog post, we discussed how to create and test POST requests.  This week, we will tackle testing PUT requests.  A PUT request is actually very similar to a POST request; the major difference is that POST requests are intended to create a new record, and PUT requests are intended to replace an existing record. 

Let's return to the Swagger Pet Store to learn how to create a PUT request.  Click on the PUT /pet request to open it:

As you can see, the endpoint and the body of the request are exactly the same as that of the POST request.  The only difference is the http verb used: PUT instead of POST.  To see how the PUT request works, let's first do a POST with these values:

{

  "id": 1,

  "category": {

    "id": 1,

    "name": "Cat"

  },

  "name": "Grumpy Cat",

  "photoUrls": [

    "https://pbs.twimg.com/profile_images/948294484596375552/RyGNqDEM_400x400.jpg"

  ],

  "tags": [

    {

      "id": 1,

      "name": "Mixed breed"

    }

  ],

  "status": "available"

}

Then do a GET to make sure that the pet was added correctly.  Now, let's do a PUT with these values:

{

  "id": 1,

  "category": {

    "id": 2,

    "name": "Dog"

  },

  "name": "Droopy Dog",

  "photoUrls": [

    "https://upload.wikimedia.org/wikipedia/en/thumb/f/fd/Droopy_dog.png/150px-Droopy_dog.png"

  ],

  "tags": [

    {

      "id": 2,

      "name": "Beagle"

    }

  ],

  "status": "pending"

}

Notice that all of the values in the body of the request have been changed except for the pet id. After you have submitted the request, do a GET to confirm that all of the values have been replaced.

A PUT request will always replace ALL of the values in the entire record, which means that if any of the values are missing, they will be removed.  For example, if we did a PUT request on that same pet id, and only included a name and a photo URL, that will be all that will be saved. Any tag or status that was present in the record before will now be gone.  

To talk about testing PUT requests, I'm going to move away from the Swagger Pet Store and use a different example. Let's consider an application that stores a list of employees.  The application writes to a table that has these values:

The PUT request that will be used to update a record in the table will look like this:

{

  "firstName": "Amy",

  "lastName": "Miller"

}

And the URL of the request will look like this: app/employee/1.  In this application, the employee id is passed in through the URL, rather than the body of the request, which is a common practice.  

The first thing we will test in our imaginary application is the Happy Path: if we send this request, does employee 1 now have the name Amy Miller instead of Fred Smith?  You could check this by querying the database directly, and by doing a GET request.  

Next, we'll see what happens when trying to do a PUT request on a record that doesn't exist.  Imagine doing the same request, but instead of using the URL app/employee/1, we'll use app/employee/3.  As you can see in the data table, there is no record 3.  So this request should return an error, such as a 404 Not Found response.  You can also try passing in ids that make no sense, such as letters, words, or symbols, and passing in no id at all.  All these requests should return an appropriate error message.

Now let's turn to the body of our request.  Let's imagine that both the firstName and the lastName are required for the PUT request.  We should get an appropriate error message if we send in an empty body, or just the firstName, or just the lastName.  In a scenario where there are many more fields than we have here, you'll want to test lots of different combinations where required and non-required fields are missing.  

As I mentioned above, a PUT request should replace ALL the values in a record; it's like the record was completely removed and replaced with a brand new one.  So if we consider for a moment a scenario where the firstName is NOT required, and record 1 is currently Amy Miller, and we do a test where we send in only a lastName of "Brown", record 1 should now have a NULL value for the firstName, and Brown should be the lastName.  

You could also see what happens when you try to pass in a request with fields that aren't in the table at all!  In this example, you could try sending in 

{

  "firstName": "Amy",

  "middleName": "Jo"

  "lastName": "Miller"

}

and verify that the field is either ignored or an appropriate message is returned.  

Now we can go on to the individual fields themselves. What sort of values are allowed? What are the character limits?  You'll want to test each individual field to make sure that you receive an appropriate error message when the limits are violated, and that you receive an appropriate error message when you try to pass in characters or values that are not allowed.  You'll also want to try sending in values that could be used for cross-site scripting or SQL injection; these should either be rejected outright or sanitized in such a way that a malicious attack would not work.  

Finally, you can return to the http verb and the general request.  What happens if you change the PUT request to a POST request?  In our hypothetical application, a POST request with a URL of app/employee/1 will return a 409 error, because record 1 already exists.  You can also see what happens if you remove or change required headers on the request.  

Hopefully this post has given you a clear indication of how PUT requests behave, and the best ways to test them.  Next week, we'll be on to the lesser-used PATCH request!  

Testing POST Requests

Last week, I introduced the concept of the GET request and how to test it.  This week we'll move to POST requests.  POST requests are perhaps the most important of the RESTful requests, because they are what adds new records to your application's database.  It's very important to test your POST requests well, because they will have a direct impact on the quality of data in your database.

To learn about POST requests, we'll once again use the Swagger Pet Store and Postman.  Navigate to the Pet Store (http://petstore.swagger.io) and click on the first POST request listed: "/pet".  This POST request will add a pet to the pet store.  Take a look at the Example Value shown:

This is the body of the POST request.  Unlike GET requests, which usually don't have a body, you will usually find some json or xml in a POST request.  The body represents the data that you are adding to the database.  Now click on the Model link:

This describes a bit about what all of the values in the body are.  You can click on the ">" icons to open up each section of the model.  I find this model to be a bit vague in terms of defining what a category is and what tags are, so I'm going to do a little guesswork.  Let's walk through each section of the Pet model:
id: this is the id of the pet, which can be used in the GET request we tested last week
category: this represents what kind of animal the pet is.  The id is the unique identifier for the category, and the name is the word describing the animal.
name: this is the pet's name
photoUrls: these are strings that link to pictures of the pet
tags: these are descriptive phrases that can be added to the pet.  The id is the unique identifier for the tag, and the name is the descriptive word describing something about the pet
status: this is the status of the pet in the store.  The status can be available, pending, or sold.

We can use the information in the model to create a POST to make a new pet.  Click on the "Try it out" link, and replace the body of the request with this information:
{
  "id": 102,
  "category": {
    "id": 1,
    "name": "cat"
  },
  "name": "Grumpy Cat",
  "photoUrls": [
    "https://pbs.twimg.com/profile_images/948294484596375552/RyGNqDEM_400x400.jpg"
  ],
  "tags": [
    {
      "id": 1,
      "name": "blue eyes"
    }
  ],
  "status": "sold"
}

Before you click the Execute button, I do want to point out something in this POST request that is different from what you will most likely experience in the applications you test.  Generally, when you post an object with an id and that id already exists in the database, you'll get a message that the record already exists.  In the case of the Pet Store, if that id already exists in the database, it will get overwritten with the object that you have posted.  

Now click the "Execute" button.  Take a look at what was returned in the body of the response.  You should see all the data you added.  It may be in a different order than it was in the body of the request, but that's not important.

Let's test to make sure that the new pet was really added!  Return to the GET pet/{petId} request, open it up, and click "Try it out".  Enter "100" into the id field, and click "Execute".  You should see the pet you added in the body of the response.  

Now that you have created a successful request in Swagger, let's try one in Postman.  Open up the Postman application, and click on the plus button (+) to create a new request.  Click on the dropdown icon next to the word "GET" and choose "POST" instead.  Enter in the request URL: http://petstore.swagger.io/v2/pet. Click on the "Body" tab underneath the URL and select the "Raw" option.  In the body section just underneath, paste the request that you used earlier in Swagger.  You may want to vary it slightly by changing the id or the pet's name.  

There's one more step we need to do before we can send this request in Postman, and that is to add a header.  Headers are used in HTTP requests and responses to pass additional information to the server.  In this case, we need to tell the server what content-type to expect.  Click on the "Header" tab underneath the URL.  In the space underneath "Key", add "Content-Type".  In the space underneath "Value", add "application/json".  In this way, we are telling the server that it should look for a body in json format when we send our request.  Now click the "Send" button.  In the bottom half of the Postman window, you should see a 200 response code, and a response body that has all of the data you added.  Now you can use a GET request in Postman to check to make sure that your pet was added.  Return to last week's blog post if you need help setting that up.  

Just as we found when we discussed testing forms, there are many different scenarios to test when testing POST requests.  First, we'll want to test many Happy Path scenarios.  Try a number of different POSTs where you vary the id, the category of the pet and its id, the pet's name, the photo URL, the tag and the tag's id, and the status of the pet. You'll want to make sure to test all three statuses of the pet: sold, available, and pending.  Also note that it's possible to pass in more than one photo URL, and more than one tag.  Here's an example of a body of a POST request where three photos and two tags are passed in:

{

  "id": 102,

  "category": {

    "id": 2,

    "name": "dog"

  },

  "name": "Snoopy",

  "photoUrls": [

    "https://schulzmuseum.org/wp-content/uploads/2017/06/920608_FlyingAce-200.jpg",

    "https://www.calmuseums.org/images/snoopy/snoopy.png",

    "https://vignette.wikia.nocookie.net/peanuts/images/2/28/AmigosperdemoSono.png/revision/latest?cb=20110823202539"

  ],

  "tags": [

    {

      "id": 2,

      "name": "beagle"

    },

    {

      "id": 3,

      "name": "flying ace"

    }

  ],

  "status": "available"

}

Now that you have tested a number of Happy Path scenarios, it's time to think about breaking the request.  First, you may have noticed in the Swagger Model that only two fields are required: the name of the pet, and one photo URL.  What happens if you pass in just those two fields?  

{

  "name": "Snoopy",

  "photoUrls": [

    "https://schulzmuseum.org/wp-content/uploads/2017/06/920608_FlyingAce-200.jpg"

  ]

}

We get a 200 response, and the pet has been added with an id that we can use to retrieve it.  What happens if there are no values in the body of the request?  What happens if one of the required fields is missing?  Now is a good time to experiment by passing in a variety of combinations of fields.  

It's worth noting at this point that you may get a 500 response for nearly every error condition you provoke.  In a real-world application, you'd get more appropriate response codes with more appropriate error messages. 

Next, it's time to take another look at passing in multiple URLs and tags.  How many URLs can you pass in before provoking an error?  How many tags can you pass in before you get an error?  

Now we'll look at the limits of the values we are passing in.  For example, what happens if you create a pet with an id of 0?  What happens if you create one with the id of -100000?  What happens if you send in an id of "FOO" or $%^?  You can test this on all of the ids: the pet's id, the category id, and the tag id.  How long are the text strings allowed to be?  How short are they allowed to be?  Are there any characters that are not allowed?  You'll want to test the upper and lower limits of the strings, and verify that any forbidden characters are not allowed.  This is also a good time to test for SQL injection and cross-site scripting.  For example, you could try passing in 

<script>alert('XSS')</script> 

or 

' or 1 = 1-- 

and see if those are allowed.  (In the case of the Pet Store, these will probably be allowed, but a real application should forbid these or at least render them ineffective.)  

Another thing to test is the enum field for the pet's status.  The pet's status should be limited to available, pending, and sold.  What happens if you pass in another word or number or symbol?  What happens if you pass in a valid status with all caps, or with some of the letters capitalized?  A POST request should generally be case-insensitive.  You will also want to test the photo URL, making sure that you can't pass in malicious scripts, and making sure that you can't pass in something that isn't a URL.

Once you have finished testing the body of the request, you can test the headers of the request.  This particular POST request requires a Content-Type header, and it will allow either an application/json header or an application/xml header.  If the header is missing, you'll get a 415 error.  If you use an application/xml header, you'll get a 400 error, unless you first change the body of the request to be in XML format.  You could also test to see what happens when you send an xml body with an application/json header.  

Finally, you can test the URL of the request itself, and the request type.  What happens if you make the request an https request?  What happens if you change the POST request to a GET?  

In summary, there are many ways to test POST requests, many of which would not be possible if you were focusing only on the UI.  In addition to testing required and non-required fields and field validation, you can also manipulate the object passed into the database and the headers and URL used to make the request. 

Next week, we'll move on to PUT requests!  

Testing GET Requests

Last week I introduced the concept of RESTful API requests, and discussed why it's crucial that we test them in our applications.  This week, we will begin our discussion of RESTful request types with the GET request.  This is usually the easiest request to test, because all we are doing is retrieving data from the database.  We don't need to worry about whether we are manipulating data correctly; we just need to retrieve it and check that we get a correct response. 

In my opinion, the two best tools to learn about REST requests are Swagger and Postman.  Swagger is an open-source framework that allows developers to create documentation for RESTful APIs.  If your API has a Swagger file, it's easy to see what kinds of requests your API allows, and what sorts of parameters those requests are expecting.  The developers of Swagger have also created a sample application that you can use to practice making REST requests:  petstore.swagger.io.  This website simulates an online pet store, where users can enter and retrieve information about their pets.

When you go to the petstore.swagger.io website, you can see that there are a variety of requests available for the pet endpoint.  Let's take a look at the GET /pet/{petId} endpoint.  Click on that request, then click on the "Try it out" button. You will notice that a petId is a required parameter.  Enter "1" into the parameter text field, and click the "Execute" button.  Scroll down to the Response Body section, and you will see that a pet has been returned!  Scroll up a bit and take a look at the Request URL: http://petstore.swagger.io/v2/pet/1. We will use this URL to learn about making requests in Postman.

Postman is the best tool that I have found for testing REST requests. It is available for free at https://www.getpostman.com, and there is also a paid Team version. If you don't have it already, download Postman and start it up.

Postman should start up with an open tab, set to use a GET request. All you have to do now is enter the request URL that we used before, and click the "Send" button. You should get the same response that you did when you made the request in Swagger.

Now that we understand how GET requests work, let's think about how to test them! We have obviously tested a Happy Path scenario where we are getting a pet with an id of 1. What happens if we change the id parameter to 2? What happens if we change the id parameter to -1? What if it's 0? What if it's "foo"? I should mention here that because the Swagger Pet Store is a test application, you may see results or behavior that would not be wanted in an application. For instance, when I tested with a parameter of -1, I got a result. It's a fairly common expectation that the ids of data objects not be negative numbers, so this would be a bug in a real application.

Let's find out what happens when we don't enter any parameter at all, so our request URL is just http://petstore.swagger.io/v2/pet. We get an xml response that says "unknown". You may also notice that the status code returned is "405 Method Not Allowed".

The status codes you get after you make a REST request tell you a bit about the behavior of your application.  We'll discuss status codes further in a later post, but for now, let's note that a status code of 200 is good, and a status code that begins with 4 generally indicates that something has gone wrong.  You may get a 404 status when you search for a pet id that doesn't exist.  This means that the record was not found. 

Let's return now to the Swagger Pet Store and take a look at another GET request.  This time, we will look at the pets/findByStatus request.  Click on the request, and then click the "Try it out" button.

Note that in order to execute this request, you need to select a status.  Let's click the "sold" status and then click the "Execute" button.  If you scroll down to the response window, you can see that many pets were returned.  Now scroll up a bit and take a look at the request URL that was used: http://petstore.swagger.io/v2/pet/findByStatus?status=sold.

The question mark in the URL indicates that we are using a query parameter.  This is a little different from the path parameter we saw when we were doing a GET by pet id.  Query parameters always begin with a question mark, and then have the parameter name, an equals sign, and the value of the parameter we want to use.  

Let's copy this request and run it in Postman.  Simply use the + button near the top of the screen to open a tab and create a new request.  It should be a GET request by default.  Enter the URL and click "Send".  You should get the same result that you saw when you ran the request in Swagger.  

We have now tested one Happy Path for this request.  You can also run the request using the "available" parameter and the "pending" parameter.  You could run this request: http://petstore.swagger.io/v2/pet/findByStatus?status=pending,sold and get all of the pending pets and all of the sold pets!  What happens if you send a value of "foo" for the status parameter?  You will get an empty set [ ] as a result.  This is different from what we saw when we sent in "foo" as the parameter for the GET pet by pet ID request.  This demonstrates the difference between a path parameter and a query parameter.  When a path parameter does not exist, you will generally get a 400-level response code.  When a query parameter does not exist, you will generally get a 200 response code and an empty set in the response body.  

Now that you have tried out two different GET requests, you can test further by manipulating the request URL in any number of ways.  For example, you could see what happens if you make the request an https request instead of http.  You could find out what happens if you change the "io" in the request to "com".  You could change the "v2" to "v1".  You could remove the "/pet" endpoint.  You could try changing the GET request type to a POST or a DELETE type.  Seeing what happens when you manipulate the URL and the request type will give you a sense of how REST requests work, and will also give you ideas for what you could test in an API that you are responsible for. 

I hope that this post has demonstrated just how easy API testing is, and also how versatile it can be.  With API testing, you can test many scenarios that would not be possible to test through the UI.  This is a valuable strategy to use in finding potential design and security flaws earlier in development.  Next week, we'll move on to POST requests!

Introduction to REST Requests

More and more companies are moving toward a microservices model for their applications.  This means that different sections of their application can have a separate datastore and separate commands for interacting with that datastore.  The advantage to this is that it's easier to deploy changes to a small component of the application rather than the entire application; it also means that if one microservice goes down, the rest of the application can continue to function.  For example, let's imagine you have a website for a bike rental service.  The site has a microservice for the reservation system, and a second microservice for the inventory.  If the microservice for the inventory goes down, users will still be able to make reservations for bike rentals, using cached data from the inventory microservice.

Most microservices are using APIs, or Application Programming Interfaces, which are a set of commands for how a service can be used.  And most APIs are using REST requests, or Representational State Transfers, through HTTP to request and send data.

Yet in spite of the common usage of RESTful APIs in today's applications, many testers do not know just how easy it is to test them!  This post will serve as a gentle introduction to REST requests for use in API testing.

Why would you want to test REST requests, rather than just wait and test through the UI?  Here are a few good reasons:

• testing REST requests means that you can find bugs earlier in the development process, sometimes even before the UI has been created!
• malicious users know how to make REST requests, and can use them to exploit security flaws in your application by making requests the UI doesn't allow
• it's easy to automate REST requests, and they run MUCH faster than UI automation (see my earlier blog post for further discussion of API vs. UI automation)

To get started in RESTful testing, first think about what you see in a URL when you navigate to a website.  For example, you might see: https://www.foobank.com/customers/login.  It's easy to see how this URL is defining what page you are navigating to:

• the https is specifying that this is a secure request
• the www.foobank.com is the domain, which says that you want to go to the Foobank website
• the customers is the first part of the path, which says that you are a customer and therefore want to go the Customers section of the website
• the login is the last part of the path, which says that you want to go the login screen

One thing that's not seen in the URL is the type of RESTful request being made.  This is known as an HTTP verb, and they are easy to understand:

• a POST request adds a new record to the database
• a GET request retrieves a record from the database
• a PUT request takes a record from the database and replaces it with a new record
• a PATCH request modifies an existing record in the database
• a DELETE request removes a record from the database

In this case, a "record" can be any section of data that is grouped together.  For example, it could be a mailing address for a customer; or it could be all of the contact information for that customer; or it could be every single datapoint associated with that customer.  It's up to the creators of the API to decide what should make up the record.  

In the next several blog posts, I'll discuss each of the HTTP verbs in detail and describe how to test them.  I am sure you will be excited to discover just how much you can test without even navigating to your application's web pages!

Testing the Login Screen

The login screen is the first line of defense between your application and a malicious unauthorized user.  But it's so easy to forget about testing the login screen!  This is because as a tester, you are in the application every single day.  Getting past the login screen is a step you take dozens of times a day to get to whatever feature you are currently testing.

Let's take some time to look at the various ways we should be testing login screens.

First, make sure that the password field masks text as you type in it.  Next, check to make sure that both the username and the password fields are expecting case-sensitive text.  In other words, if your username is "FOO" and your password is "bar", you should not be able to log in with "Foo" and "Bar".

Next, find out what the validation requirements are for the username and password fields.  How short can the values be?  What's the longest they can be?  What characters will be accepted?  Verify that the length requirements are respected in both fields, then test with every non-accepted character that you can think of.  Be sure to test with non-UTF-8 characters and emojis as well.  You may have heard a story several years back of a bank customer who broke the application by setting her password to have an emoji in it!

Also be sure to test for SQL injection.  For example, if you enter 1' or '1' = '1 into the password field, you may be telling the database to return 'true' and allow logging in, because of course '1' always equals '1'.

Now let's test with every possible combination of username and password states.  Each field can be either empty, incorrect, correct but with the wrong case, and correct.  This generates sixteen possible combinations:

It may seem like overkill to test all these combinations, but I have seen situations (fortunately that were only in the earliest stages of development) where I could log in with an empty username and password, or log in where both the username and password were incorrect!

Another helpful thing to test is putting an empty character or two at the beginning or end of the fields.  When the user submits their login request, empty characters should be stripped from these values.  If this does not happen, the user may wind up with a situation where they have typed their correct password: "bar ", and the password is not accepted because of the space at the end.  This is very frustrating for an end user who doesn't know the empty character is there.

Next, let's think about the kind of error message you are getting when you try to login with invalid credentials.  You don't want to give a malicious user any kind of hints about whether they have guessed a username or a password correctly.  If the malicious user types in "FOO" for the username and "password" for the password, you don't want to return a message that says "Invalid password", because this will let the malicious user know that the username is correct!  Now all they have to do is keep the username the same and start guessing at the password.  It is much better to return a message like "Invalid credentials".

Now let's take a look at what is passed into the server when you are making a login request.  Open up the developer tools for the browser you are using, and watch the request that goes through.  Do you see the username or the password in clear text anywhere?  If you do, this should be fixed!  Usernames and passwords should both be encrypted in the request.  You can also check the request by using an interception tool like Fiddler or Burp Suite.

Once you have logged in with correct credentials, take a look at the cookie or session id that has been set for your user.  Do you see the username and password anywhere?  All cookies, web tokens, and session ids should not include the user credentials, even if they are encrypted.

Finally, be sure to test logging out!  When you log out, the username and password fields on the login screen should be cleared, unless there is a feature to remember the credentials.  If there is such a feature, the password should definitely be masked and should give no indication of how many characters are in it.  Upon logging out, any session ids or web tokens should be deleted.  If there is no feature to remember credentials, the cookie should be deleted as well.  There should be nothing that you can grab from developer tools or interception tools to use to pretend that you are logged in.

This is a lot to test; fortunately, we can take advantage of automation for regression testing once we have done an initial test pass.  Here are a few ideas for what to automate:

API login calls using all the combinations of username and password listed above
UI logins with too many characters and invalid characters in the username and password fields; verify that the error message returned is the correct one
Visual testing of the login screen to verify that the password is not displayed in clear text (a good tool for visual testing can be found at https://www.applitools.com)

Security is such an important part of any application!  By running through all of these login scenarios, you can help bring peace of mind to your product team and to your end users. 

Testing Back Buttons

The back button is so ubiquitous that it is easily overlooked when it comes to web and application testing.  The first thing to know when testing back buttons is that there are many different types.  The two major categories are: those that come natively, and those that are added into the application.

For native buttons, there are those that are imbedded in the browser, those that are imbedded in a mobile application, and those that are included in the hardware of a mobile device.  In the photo above, the back arrow is the back button that comes with the Chrome browser.  An Android device generally has a back button at the bottom that can be used with any application.  And most iPhone apps have a back button built in at the top of every page.

Added back buttons are used when the designer wants to have more control over the user's navigation.  In the example above, the Home button is used to go back to the W3 Schools home page.  (I should mention here that W3 Schools is an awesome way to learn HTML, CSS, Javascript, and much more: https://www.w3schools.com/default.asp)

When you are testing websites and applications, it's important to test the behavior of ALL of your back buttons, even those that your team didn't add.  The first thing to do is to think about where exactly you would like those buttons to go.  This seems obvious, but sometimes you do not want your application to go back to just the previous page.  An example of this would be when a user is editing their contact information on an Edit screen.  When the user is done editing, and has gone to the Summary page, and they click the Back button, they shouldn't be taken back to the Edit screen, because then they'll think they will have to edit their information all over again.  Instead, they should go back to the page before.

Another thing to consider is how you would like back buttons to behave when the user has logged out of the application.  In this case, you don't want to be able to back into the application and have the user logged in again, because this is a security concern.  What if the user was on a public computer?  The next user would have access to the previous user's information.

For mobile device buttons, think about what behavior you would like the button to have when you are in your application.  A user will be frustrated if they are expecting the back button to take them elsewhere in your app, and it instead takes them out of the app entirely.

If your application has a number of added back buttons, be sure to follow them all in the largest chain you can create.  Look for errors in logic, where the application takes you to someplace you weren't expecting, or for memory errors caused by saving too many pages in the application's path.

You can also check whether the back button is enabled and disabled at appropriate times.  You don't want your user trying to click on an enabled back button that doesn't go anywhere!

In summary, whenever you are testing a web page or an application, be sure to make note of all of the back buttons that are available and all of the behaviors those buttons should have.  Then test those behaviors and make sure that your users will have a positive and helpful experience.

Automated Form Testing

Now that we have looked at all the different ways we can manually test forms, it's time to think about automating those tests so you don't have to run through all the tests again every time you need to do a regression test!  But before you jump into automation, think for a while about what tests you really want to run.  Let's take last week's form as our example again.

What sorts of things would we want to make sure still work when we do our regression test?

• We'll want to make sure that every field can be populated with data and saved
• We'll want to make sure that all of the required fields are still required, and that we get an appropriate error message when we leave a required field blank
• We'll want to make sure that validation rules are respected in each field
• We'll want to verify that both the Save and the Cancel buttons work correctly

As I've discussed in a previous post, automated tests involving the GUI can often be slow and flaky.  So we'll want to limit the number of times we spin up a browser for testing.  But we'd also like our tests to only assert on one thing at a time.  Here are my suggestions for a suite of tests that could be run on this form.  I've written them in Cucumber for ease of reading:

Scenario: Happy Path

Given I am adding a new user

When I fill out all the fields and click the Save button

And I navigate to the Users page
Then I verify that all of the fields are displayed on the page

Scenario: Required Fields

Given I am adding a new user

When I leave all fields empty

And I click the Save button

Then I verify that an error message is present for each required field

Scenario: Validation Rules

Given I am adding a new user

When I give each field an invalid value

And I click the Save button

Then I verify that an error message is present for each invalid field

Scenario: Cancel Button

Given I am adding a new user

When I give each field a value

And I click the Cancel button

Then I verify that all fields are now empty

So, we can cover a lot of territory with just four automated tests!  

Our Happy Path tests that values in all of the fields will save correctly, and it also tests the Save button.  If one of the values does not save, we will know we have an error.  If all of the values do not save, we will know there is a problem either with the Save button or with the underlying data store. 

Our Required Fields test checks to make sure that every field that is required displays an appropriate error message if it is not filled out.  If even one required field does not display the message, we will know there is a problem.

Our Validation Rules test violates one rule for each text field.  It would be a good idea to mix and match the different types of rule violations.  For example:

First Name- send a value with numbers and non-letter characters, such as $23.00

Last Name- send a value with not enough letters, such as A

Personal Phone- send a value with the wrong number of digits, such as 008675309

Work Phone- send a value with non-number characters, such as 800*555-!@#$

Street Address 1- send a value with way too many characters, such as all the lyrics to "Frosty the Snowman" (my favorite song to test with)

Street Address 2- send a value with one too many characters

City- try sending a script, such as <script>alert("XSS Here!")</script>

State- send an invalid state, such as XX

Zip Code- send a Zip Code with the wrong number of digits, such as 0377

While this does not test every single way that the rules could be violated in every text field, it covers a wide variety of possibilities, and verifies that for each error, you receive an appropriate error message.  You could always add more tests here if you felt that more coverage was needed.

Finally, we verify that the Cancel button works correctly.  

With just these four automated tests, we're able to make sure no functionality is broken.  Running these regression tests will free you up to do more exploratory testing in your application, and to test new and more interesting features!

Testing Forms

Today is the day that we'll be putting it all together!  In the past few posts, we've been looking at different types of text fields and buttons; now we'll discuss testing a form as a whole.

There are as many different ways to test forms as there are text field types!  And unfortunately, testing forms is not particularly exciting.  Because of this, it's helpful to have a systematic way to test forms that will get you through it quickly, while making sure to test all the critical functionality.

Let's take a look at the form below, and I will walk you through a systematic approach to testing it.  For simplicity, we'll assume that the users will all be US-based.  (See my previous posts for discussions on testing international postal codes and phone numbers.)

Step One:  Required Fields

The first thing I do when I test a form is to make note of which fields are required.  This particular form denotes required fields with a red asterisk.  We can see that First Name, Last Name, Personal Phone, Street Address 1, City, State, and Zip Code are all required.  I want to verify that the form cannot be submitted when a required field is missing and that I'm notified of which field is missing when I attempt to submit the form.
1. I click the Save button when no fields have been filled out. I make sure that all of the required fields have error messages.
2. I fill out the two non-required fields, click the Save button, and verify that all the required fields have error messages.
3. I fill out one required field and click the Save button, and I verify that all the other fields have error messages.  I start with submitting just the First Name, then just the Last Name, then just the Personal Phone, etc., until I have cycled through all of the required fields.
4. I try various combinations of two, three, four, and five required fields to make sure the appropriate error messages are displayed.  I don't concern myself with testing every possible combination, since it's highly likely that the required field will behave similarly in these situations.
5. I try filling out all the required fields but one, and verify that I still receive an error for that field.  I cycle through all the possibilities of having one missing required field.
6. I fill out all of the fields- required and non-required- except for one required field, and verify that I still receive an error.
7. I fill out all of required fields and click the Save button, and I verify that no error messages appear and that the entered fields have been saved correctly to the database.
8. Finally, I fill out all of the fields- required and non-required, and I verify that no error messages appear and that the entered fields have been saved correctly to the database.

Step Two: Field Validation

The next thing I do is verify that each individual text box has appropriate validation on it.  First, for each text field, I discover what the upper and lower character limits are.  I enter in all the required fields except for the one text box I am testing.  Then in that text box:
1. I try entering just one character
2. I try entering the lower limit of characters, minus one character
3. I try entering the upper limit of characters, plus one character
4. I try entering a number of characters far beyond the upper limit
In all of these instances, the form should not save, and I should receive an appropriate error message.
Next:
5. I enter the lower limit of characters, and verify that the form is saved correctly
6. I enter the upper limit of characters, and verify that the form is saved correctly

Now that I have confirmed that the limits on characters are respected, it's time to try various letters, numbers and symbols.  For each text field, I find out what kinds of letters, numbers, or symbols are allowed.  For example, the First and Last Name fields should allow apostrophes (ex. O'Connor) and hyphens (ex. Smith-Clark), but should probably allow no numbers or symbols.  For each text field:
1. I try entering all of the allowed letters, numbers and symbols
2. I try entering the letters, numbers, or symbols that are not allowed, one at a time, until I have verified that they are all not allowed

For fields that have very specific accepted formats, I test those formats now.  For instance, I should be able to enter a Zip Code of 03773-2817, but not one of 03773-28.  For the State field, the two-letter code I put into the field should be a valid state, so I should be able to enter MA, but not XX.

Even though I've already tested all the forbidden numbers and symbols, I try a few cross-site scripting and SQL injection examples, to make sure no malicious code gets through.  (More on this in a future post.)

Step Three: Buttons

Even though I have already used the Save button dozens of times at this point, there are still a few things left to test here.  And I have not yet used the Cancel button.  So:
1. I click the Save button several times at once, and verify that only one instance of the data is saved
2. I click the Save button and then the Cancel button very quickly, and verify that the data is saved and that there are no errors
3. I click the Cancel button when no data has been entered, and verify that there are no errors
4. I click the Cancel button when data has been entered, and verify that the data is cleared.  I'll try this several times, with various combinations of required and non-required fields.
5. I click the Cancel button several times quickly, and verify that there are no errors

Step Four: Data

Finally, it's time to look at the data that is saved.  In earlier steps, I've already done some quick checks to determine if the data that I am entering is saved to the database.  Now I'll check:
1. That bad data I've entered is not saved to the database
2. That good data I've entered is saved to the database correctly
3. That good data I've entered is retrieved from the database correctly, and displayed correctly on the form

If the form I am testing has Edit and Delete functionality, it's important to test these as well, but I've covered these in my post CRUD Testing Part II- Update and Delete.

Once I've gone through a form in this systematic way, it's almost certain I will have found a few bugs to keep the developers busy.  It's tedious, yes, but once the bugs have been fixed I can move on to automation testing with confidence, knowing that I have really put this form through its paces!

Testing Buttons

Buttons tend to be something that it's easy to forget about.  The "Save" button is so ubiquitous that it seems like it would just work.  But overlooking testing buttons on a page can also mean overlooking bugs.  Recently someone told me about new functionality she was testing on an existing web page.  The new feature worked great, but her team forgot to test the "Delete" button.  It turned out that the developers had forgotten to account for the delete action in their new feature, and now "Delete" did nothing! 

Here are a few things to think about when testing buttons:

1. Test the "happy path" of the button.  Usually buttons have some sort of message on them that tell you what they are supposed to do.  So try it out, and make sure that the button delivers on its promise!  Did the "Save" button really save your data?  Did the "Delete" button delete it?  Did the "Clear" button clear it?  Did the "Search" button execute a search?  (Note: a "Back" button is also common, but back buttons are tricky enough that I will be saving them for a separate post.)

2. Misuse the button.  For example, quickly press the "Save" button twice when adding data.  Were two records saved instead of one?  Does a button get confused when you press it twice?  What about when you quickly press one button and then another?  One of the most amusing bugs I have found in my career was a "Refresh" button that redrew the screen, making the button larger whenever I pressed it.

3. Is the button there when it's supposed to be?  When we have spent a lot of time testing an application, it's easy to get used to the page and not notice when things are missing.  Think about your user stories when looking at the page.  What would your user need when doing certain activities on that page?  Follow the path that the user would take, and check for the buttons as you go.

4. Think about when the button is enabled and when it is disabled.  Does it make sense?  For example, is the "Save" button only enabled when all the required fields on your form have been filled out?  Is the "Clear" button only enabled when a field has been dirtied?  How do you know the button is enabled?  Can you tell by looking at it?  Does the button look enabled when it isn't?  Does the button look disabled when it's really active?  What happens when you press the button and it's not enabled?  Do the enabled and disabled rules for the button make sense?

5. Finally, see if you can hack your buttons!  For example, if you have a "Save" button that is disabled on your form because some of the required fields are missing, can you edit the html on the page so that it is enabled, and can you then use the button?  If you view the html and you see that there is a hidden button on the page, can you make it visible and active?  At best, that bug is an annoyance that will get inaccurate data into the database.  At worst, the bug represents a way that a hacker can infiltrate your system!  Imagine that you have a button that should only be visible and enabled if a user is an administrator.  If a malicious user can make that button appear on the screen and be active, they will have access to pages or features that only an admin should have. Your developer should include checks whenever a button is clicked on to make sure that the user has the rights to do what the button does. 

If you have never edited the html on a web page while testing, here are a few simple instructions for the Chrome browser:
a. Click on the three-button menu on the top right of your screen, and choose "More Tools -> Developer Tools".  A new panel will appear on the bottom or the right side of your screen.
b. Right-click on the button you want to test, and click "Inspect".  In the developer tools panel, you will now see the html for that button highlighted.
c. Right-click on that highlighted text and choose "Edit as HTML".  An editable text window will open up. 
d. If you see text such as "'disabled'=disabled" delete the text.  Click away from the editable field, and see if your button is now enabled. If it is, click on it and see what happens!
e. To find hidden buttons, look at the html in the developer tools, and use the search bar to search for "button".
f. If you find a button with the markup "ng-hide", try changing it to "ng-show".  See if you can get the button to appear on the page!

Buttons are one of the most important things to test.  Just imagine a user's frustration if the button they are trying to use is disabled, or doesn't do what they expect it to.  By being diligent in our testing, we can ensure that our users will want to work with our application. 

Testing Phone Fields- Part II- International Phone Numbers

If I were to be given three wishes for the world, my third wish (after world peace and ending world hunger) would be to standardize international phone numbers, because testing them is so complicated!  Let's take a look at some phone number patterns from around the world:

Mexico:
Phone numbers are ten digits, plus an area code of either two digits or three digits.  So the local number will be either twelve or thirteen digits.

Italy:
Landline numbers are generally nine to eleven digits, but some can be as short as six.  Mobile numbers are usually ten digits, but there are also some nine-digit numbers.

Japan:
Telephone numbers in Japan have an area code, an exchange number, and a subscriber number.  Area codes can have between two and five digits.  Generally the length of the entire number is limited to nine digits, so if the area code is longer, the exchange and subscriber numbers will be shorter.

Now let's think about international calling codes.  Calling codes can range from one digit (such as the US's +1) to three digits (such as +351 for Portugal).  So between the international calling codes and the phone number itself, there is a wide range of the number of digits that might be expected for a phone number.

Finally, let's think about number separation.  While US numbers are always separated in a 3-3-4 pattern, in other countries the numbers can be grouped in various ways.  Here are just some of the ways that phone numbers are grouped in England:

(xxx) xxxx xxxx
(xxxx) xxx xxxx
(xxxxx) xxxxxx
(xxxx xx) xxxxx

Therefore, it is extremely difficult to validate on whether the pattern of numbers, parentheses, and dashes entered by a user is correct.

So how on earth are we to validate international phone numbers?  The most important thing to do is to make sure that the developer doesn't try to come up with a validation regex on their own- that will make them (and you) go crazy!  Fortunately an international phone number formatting standard called E.164 has been developed.  Many companies such as Microsoft and Google have come up with regex patterns that can be used for E.164 validation.  For testing, there are free websites that will let you know if an international phone number is valid.  Most of these sites will require some sort of registration.  You can come up with a list of international numbers to test, verify with the website to see if they are valid, and then try them in your application.  For negative testing, simply use a sampling of invalid numbers and verify that you get an appropriate error message.

What format should a user use when entering in their phone number?  Some users might want to put parentheses, spaces, or dashes into their number.  Then there is the plus sign, which is used to indicate a country code.  For simplicity's sake, it would be best to expect nothing but digits and a plus sign.  It would also be good to prompt the user for what format the application is expecting.  For example, in the United States you might have a message such as: "Enter your phone number with no dashes, parentheses, or spaces.  For numbers outside the US, a plus sign (+) and the country code should be used."  Then for validation, if the number does not have a plus sign, simple US number validation rules can be applied, and if the number does have a plus sign, international validation rules can be applied.

Similarly, you may want to store the number in the database with the plus sign, to indicate that the number is an international one.  How should the numbers be displayed when retrieved in the application?  For many countries it's nearly impossible to know how the numbers should be spaced, unless the validation code you are using returns that to you.  If your country has a standard number formation, you can display the number in that standard.  If your country does not have a standard formation, or if you are displaying a number from outside your country, you can simply display the number as is, such as: +442073238299.

When it comes to international phone numbers, there really are no easy answers, which is why I am wishing for world standardization!  There are other ways to (mostly) solve the problem, such as 1) expecting only between 6 and 20 digits, not validating the number beyond making sure that it's within that number of digits, and displaying the number as entered, or 2) having a separate field for the country code, validating the number for that specific country, and formatting the number according to that validation.  What is most important is that you communicate closely with the developer to decide on a clear strategy and test it accordingly.