Essential Tools for Building RESTful APIs

Restful APIs are a great way to connect systems together. They are platform-independent well-established standards that are designed to support high performance, massive scale, and secure communication.

Will discuss delivering better REST APIs with the free tools that are already in the market, in this document I will be platform independent so weather your  Java, Asp.NET, php, Ruby or node developer the tools will still be applied.

Design your API:

Collaborative design:

The shape of the API, typically that involves 3 way discussions between the architects or engineers that are representing the API client, the Restful API itself and data provider. (On minor projects, those roles can be done in one team or even one person)

To make the solution solid, there is usually an agreement to achieve. To make sure all those different interests are happy. The client may want data that does not exists in the provider or vise versa, the provider needs data from the client that is not available.

Starting with a solid and mutual understanding of whats required is a must to achieve a successful API.

buildingRestApi

In this examples I will use JSON format, which is the most widely used format to any API nowadays, plus JSON doesn’t inherently has a schema, which gives it a lot of flexibility, different kind of Clients can use the same API and only read the fields that they are interested in.

Different kinds of Clients can read the same Schema and only read fields that they are interested in.

Tools to help us design and document the API:

Documentation is Important, Apiary helps in that… An API is only as good as its documentation. The docs should be easy to find and accessible. Most developers will check out the docs before attempting any integration effort.
The docs should show examples of complete request/response cycles.

Apiary

Build beautiful APIs Collaborative design, instant API mock, generated documentation, integrated code samples, debugging and automated testing.

You can register to Apiary with any email address or use your own Github account, if you don’t have Github account you should get one. When you signup Apiary creates for you a sample app which supports you to make notes.

How Apiary works?

Apiary Goodness, it allows you to build a design with Collaboration and Access while it gives the possibility to directly stub the API and inspect its traffic, you can connect your API design to Github which gives you the opportunity to have version control of your design.

 

API Design Alternatives:
Swagger & RAML

Swagger is a tool that it documents your API from the code, its powerful and never out of sync. the problem that you may face with swagger you need to be a developer while with Apiary you can design your Api test it without having to write code.

Raml is relatively new product, I did some experiments with RAML its cool and it has lots of powerful options.
If you are planning to build a huge API, I think Raml is a very good place to start.

My preference in designing Api’s goes to Apiary, feature-rich, open Easy to use, quick to learn and setup with very little effort… any non technical guys would understand it and see the opportunities and the benefits from the API.


 Test your API:

One of the most important tool for Restful API, which is the delivery tool HTTP.

Rest sits on top of HTTP, and HTTP sits on TCP/IP, to test your API you need to be able to test your API and see whats happening on those different levels between the Rest client, HTTP and TCP/IP.
resthttptcp

Testing Rest Client:

My favorite tools to test my API i use POSTMAN, to use POSTMAN you can install it click here, with postman you can create environments and select multiple environment and build collective tests, in this way you can build your own end to end testings and allow POSTMAN to run on the whole API.  You can program your e2e tests with JavaScript, which makes it very easy to build all sort of scenarios.

The API can be shared between the team in a click of a button, postman is a great rest client and can run on any platform that has chrome.
Postman itself is free, with very cheap upgrade you can get a collection of tests and automate your API with collection runner.

The other alternatives are RestClient on Firefox, its a very good plugin which lets you do basic alternative to postman if your not running chrome.

cURL is a very powerful tool that gives you lots control, its a great tool but it would require more effort then postman, curl command line does not run on any browser you can run it on any platform in command line Linux/Max/Windows. cURL is powerful, but if you want to make automated jobs with cURL, you need to have intermediate knowledge in that matter.

Testing HTTP Debuggers:

Web debuggers are very important, you can test your API with Burp Suite, when using Burp Suite you can test all your HTTP communication between the the browser and your Api, Burp is good in many intercepting testings, but one of the most powerful tools Burp can do is SECURITY.

Burp has Intruder, with Intruder you can test your API security, validate and sensitize any data.

Use Burp to do some deep background testing for Proxy, Repeater and DNS testing of your API, but whats definitely worth looking at is the attack testing with Spider and Intruder.Burp has a free version, and will give you everything you need to fetch your api in any imaginative way.

The other alternatives specially in the Windows world Fiddler is free web debugging proxy for any browser, system or platform.
Fiddler is great tool, it has powerful options.

I use both Fiddler and Burp, both are great and gives you lots of insight on your application.

Testing Packet Sniffers:

This is the deepest level, of monitoring your network… packet sniffers monitors your network and keeps a copy of all the tcp packets that pass through, that includes low level information that get sent between client and server.

One of the great tools for that is WireShark, its a free tool.

Using packet sniffers can come handy, but as i said its very low level information in many cases you wont be needing them to build an API.

My preference on testing goes to POSTMAN, but always it’s needed to test the security of your API and the vulnerabilities.

 


HTTP And Caching:

Checking the load testing for your API, and be confidence of how many clients your API supports.

Caching can be implemented on many levels in your application or outside your application:

HTTP caching isn’t only useful for long lived data, the cache responses maybe set to live for just 1 minute before the proxy returns to the server to refresh them but that means if you have 1000 responses per second, your API actually services one request per minute from the proxy that’s one API request for every 60k client requests.

Having cache control with max-age values to any long lived data is a very good practice to have a very fast API service, expiration caching gives you the biggest win because it saves network traffic so clients loads more quickly and API has fewer requests to deal with, but it is also dangerous because once its cached clients will use it until it expires. More importantly expiration caching is useful for shared resources, for personalize resources HTTP client wont cache the responses.

When you cant accept the client to have out of sync resources and have personal resources, you can use Validation caching instead, if your API returns ETag or last modified, the HTTP will make conditional back request, then the API has to decide if the client request has changed or not, in case it was not changed the API will return 304 not modified.

Caching in HTTP is very simple and it can make a huge difference to how much load your API can handle, without caching every single client request has to go to the server to get its response, with caching clients and proxies has its own caches. Fewer calls to your servers that means you can serve more clients more quickly.

Any shared resources, it should use expiration caching, while validation caching must be used for personal resources, make sure to have all the validation caching keys in memory database which would make it faster to serve the client.

API caching, despite what language your using JAVA, C#, Scala, node or PHP explore REDIS  Redis is an open source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs.

Any data despite its complexity can be cached in memory, caching inside your business layer is very important to allow your API communicate fast with your data layer, despite what sort of SQL or data source your using.

This subject is more advanced, but if your API needs a boost then REDIS is the answer, REDIS has different plugins for all languages and it’s pretty simple to use specially with AOP architecture.

DNS Advanced Caching, using DNS resolution with ROUTE 53 on Amazon and CloudFlare is a perfect tool to make your DNS fast and clear to stay highly available, I’m a big fan of CloudFlare and I use it for different source of projects I deal with, CloudFlare is free and provides hundred of DNS options and one of them SECURING YOUR API with HTTPS for free.

Caching and DNS are key parts of the HTTP protocol, and making good use of them gives you a lot of flexibility to provide better API performance and reliability.

SSL & DNS Always use SSL communication, this is a MUST, maybe you think your API is not important to be secured but think of all the users who are using your API and they may have the same username and password everywhere.

If you want a free SSL encryption between Clients and your first layer DNS you can use CloudFlare its free and you can use full SSL communication between Client and Cloudflare communication then Cloudflare will communicate with your servers via HTTP this would support half the way encryption.

sslThis blog uses CloudFlare SSL certification, and caching for FREE.


API Load Testing:

Performance testing our API with load test tools, all load testing tools send lots of requests to your api seeing how your API copes and using that information to design your production infrastructure.

Luckily we don’t have to setup multiple machines anymore to do any load tests, we have load testing tools that help us to do that sort of tests.

My favorite is loader.io
loader-io
You can use loader to run different type of load tests, average load, peak load or way over higher peak load. Typically higher number of users will increase the response time and that’s logical because your API is busy handling requests.

With loader you can scale your infrastructure if your using Amazon, Azure or your own servers and retest till your satisfied. Baseline performance and stress testing is just a case of creating test configuration in loader and running them frequently.

Soak testing is needed with each major release, you can define those testing with loader and run soak tests for 12 hours or more, depending on your needs.

Building such scripts with curl is pretty easy, on your Linux or windows machine… this sort of testing are recommended if you use Continuous Integration like Team City or Jenkins.

loader.io is Free of Charge and has paid services.

An alternative to loader.io you can look at blitz.io, it is not free you can use it to get more details about your services and errors. blitz make it easy to specify your loads with locations and sort of response times under different loads and locations.

Scaling tests is very important, without having performance tests you wont know what your capable of when your under pressure on production. This sort of tests will allow you to measure your Stability while keeping a good figure about speed and predictability of your API.

I wont speak about gatling and jmeter as they require your own premises, and sort of tests we need for basic API we wont need other options.

You should consider load and performance testing to be a necessary part of API delivery, It is practically negligent to deploy a Rest API without knowing exactly how it performs.


Monitoring: (Logging/Analytic/Diagnostics)

Monitoring your API with some tools that makes it easy to look whats going on inside your API and help you diagnose problems, there is growing industry that provides services that can help you analyze your Api.

One nice tool to use low level logs is loggly, its a cloud based and has a free tier of 200mb every day of log data for 7 days, that’s a very fine amount… you can upgrade to premium storage to receive more data.

Loggly has plugins to most of the plugins, and supports rest endpoints.

Loggly the world’s most popular cloud-based, enterprise-class log management service.

With loggly you can centralize your logs, and query all the log files basically it helps you identify any issues in your system.

Logging is only one part of monitoring, its the low level monitoring which tells you what your API doing.

One of the best Monitoring/Logging is newrelic:

With newrelic free you can have a full view of your application, the newrelic agent understand .net/java/php and all other languages, seeing the patterns of new relic you will understand most of the issues you were facing.

The most useful application for API is APM (Application Performance Monitoring) from newrelic, it gives you performance and breakdown with profiling sessions at code level, you get all information and where does your application waits on external apps.

I’m using newrelic since years, and for the most of the time it gave me lots of information I need, newrelic tools are awesome and they can tell you in details about your application.

Finally having a diagnostic endpoint for your API is very important to give you a view if any problems are occurring on your API.
healthcheck


For more information you can view essential rest tools website.