Creating a Demo API

I wanted to learn more about the open API specification. Specifically, 3.0.3 and a little bit of 3.1 for future reference. The most common demo API is the pet store specification. The pet store specification is daunting to teach and communicate in a short time frame. I want to show my stakeholders the API composition step as part of the new project creation. In our self-service portal, every project has a link that directs users to access an internal SwaggerHub instance. They can optionally iterate on their API contract. Once the contract is complete the users can click on the API version and click publish. Publishing fires a webhook that updates the git repository with the contract changes on an orphan branch. The developers can then merge in the controller changes. Including any data-models, they’ve defined for their project. The benefit of focusing on the API contract is standardization and time savings around the API parameters.

Finding a Demo API

I wanted to find something lighthearted then I could show and people could understand. I settled on the dog and cat APIs. Unfortunately, I couldn’t find an open API spec I could pull down. Instead, I downloaded the postman collections the websites offer. I used an npm NodeJS package to convert the postman collections to an OpenAPI specification format. On further investigation, I learned that both of these APIs are quite different from each other. Both APIs had enough data to help me get started.

Making the API simpler

I wanted to keep as much as I could from each of the APIs. I decided to cut out features that seemed to add too much challenge. I ended up with this contract. The job of this API is to retrieve information the pet breeds. Either as a complete list or by individual breed names. Another feature is to get images of those pets and upload them to the service.

Swaggerhub Code Generation Surprises

The benefit of the SwaggerHub/OpenAPI Code Generation is the time savings of that generated code. The downside is that the code is optimized for generation and not human readability. Each language implements generating from the specification differently. Thus it is common to hear people displeased by what they see. I know I was surprised when my models were not listed correctly in the controllers. The generator punted with a few empty List<> strung around.

Populating the Database

The cat and dog APIs had a fair amount of data to use off-the-shelf to populate into a database. A word of warning that the C# are snippets ran in Linqpad.

var client = new RestClient("https://api.thedogapi.com/v1/breeds?limit=10000&page=0");
client.Timeout = -1;
var request = new RestRequest(Method.GET);
request.AddHeader("Content-Type", "application/json");
IRestResponse response = client.Execute(request);
JArray dogs = (JArray)JsonConvert.DeserializeObject(response.Content);
var dogbreeds = new List<breeds>();

foreach (JObject dog in dogs)
{
	breeds d = new breeds() {
		name = dog.GetValue("name").ToString(),
		bred_for = dog.GetValue("bred_for")?.ToString(),
		breed_group = dog.GetValue("breed_group")?.ToString(),
		life_span = dog.GetValue("life_span")?.ToString(),
		temperament = dog.GetValue("temperament")?.ToString(),
		height = dog["height"]["imperial"]?.ToString(),
		weight = dog["weight"]["imperial"]?.ToString(),
		origin = dog.GetValue("origin")?.ToString(),
		country_code = dog.GetValue("country_code")?.ToString(),
	};
	dogbreeds.Add(d);
}

var dbContext = this;
dbContext.breeds.AddRange(dogbreeds);
dbContext.SaveChanges();


var client = new RestClient("https://api.thecatapi.com/v1/breeds?limit=10000&page=0");
client.Timeout = -1;
var request = new RestRequest(Method.GET);
request.AddHeader("Content-Type", "application/json");
IRestResponse response = client.Execute(request);
JArray cats = (JArray)JsonConvert.DeserializeObject(response.Content);
var cbreeds = new List<breeds>();

foreach (JObject c in cats)
{
	JToken res;

	breeds d = new breeds() {
		name = c.GetValue("name").ToString(),
		life_span = c.GetValue("life_span")?.ToString(),
		temperament = c.GetValue("temperament")?.ToString(),
		weight = c["weight"]["imperial"]?.ToString(),
		origin = c.GetValue("origin")?.ToString(),
		country_code = c.GetValue("country_code")?.ToString(),
		cfa_url = c.GetValue("cfa_url")?.ToString(),
		vetstreet_url = c.GetValue("vetstreet_url")?.ToString(),
		vcahospitals_url = c.GetValue("vcahospitals_url")?.ToString(),
		wikipedia_url = c.GetValue("wikipedia_url")?.ToString(),
		description = c.GetValue("description").ToString(),
		alt_names = c.GetValue("alt_names")?.ToString(),
	};
	cbreeds.Add(d);
}
var dbContext = this;
dbContext.breeds.AddRange(cbreeds);
dbContext.SaveChanges();

I wanted to make this API a little bit more attractive by offering more images. I found the Stanford Dog Image Dataset and the Oxford-IIIT Pet Dataset. These were great for finding sets of images. I could upload them into an object storage system. I then could reference the pictures with their size for future webpage rendering from the database.

I settled on the below schema. I cheated and made the image a string instead of an object. When I uploaded the image metadata to the database, I created an image table with height, width, id, url, and pet_type to load all the images related to a type of animal.

CREATE TABLE [dbo].breeds (
id INT PRIMARY KEY IDENTITY (1, 1),
name VARCHAR (50) NOT NULL,
life_span VARCHAR (50),
temperament VARCHAR (200),
origin VARCHAR (50),
country_code VARCHAR (50),
height VARCHAR (50),
weight VARCHAR (50),
image VARCHAR (100),
bred_for VARCHAR (100),
breed_group VARCHAR (50),
alt_names VARCHAR (100),
cfa_url VARCHAR (100),
description VARCHAR (500),
vcahospitals_url VARCHAR(100),
vetstreet_url VARCHAR (100),
wikipedia_url VARCHAR (100),
pet_type VARCHAR(10)
);
CREATE TABLE [dbo].images (
id INT PRIMARY KEY IDENTITY (1, 1),
height int,
width int,
image_url,
pet_type
);

Linqpad saves the day because I can interact with the database like code using entity framework under the hood. Formatting a query into SQL is possible but annoying when I can do something simple as code.

var dbContext = this;
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
	NewLine = Environment.NewLine,
};

var pets = new List<dogs>();
using (var reader = new StreamReader(@"C:\Users\Alan.Barr\Downloads\pets.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
	var records = csv.GetRecords<dogs>();
	foreach (var record in records)
	{

		if(record.pet_type == "dog") {
		var pet = new dogs(){
			name = record.name,
			life_span = record.life_span,
			temperament = record.temperament,
			origin = record.origin,
			country_code = record.country_code,
			height = record.height,
			weight = record.weight,
			image = record.image,
			bred_for = record.bred_for,
			breed_group = record.breed_group,
			//alt_names = record.alt_names,
			//cfa_url = record.cfa_url,
			description = record.description,
			vcahospitals_url = record.vcahospitals_url,
			vetstreet_url = record.vetstreet_url,
			wikipedia_url = record.wikipedia_url,
			pet_type = record.pet_type
		};

		pets.Add(pet);
		}
	}

}

dogs.AddRange(pets);

dbContext.SaveChanges();

Once my data was in place, I could move forward with iterating on my API project. I can use the SqlServer client directly, but, I have the pain of manually mapping my objects to C# objects. I figure most C# teams are using Entity Framework Core for most projects. I installed the entity framework core global .net tool and scaffolded my models from the database.

dotnet ef dbcontext scaffold "Data Source=DATABASECONNECTION;Initial Catalog=Platforce;integrated security=True;" Microsoft.EntityFrameworkCore.SqlServer -o Models

I removed the models I didn’t need and then spent a bunch of time fiddling with the database context file generated to allow the base object and the subtypes of cat and dog breeds. I also relearned that my database context objects needed to be added as transients during the project start-up. My last hurdle for this API was implementing the image uploading feature. I wasted a lot of time trying to detect what image was what kind. I ended up stipulating in the API that I accepted either PNG or JPG and uploaded based on that information to an object storage endpoint.

Conclusion

There is merit in generating code if you work on many APIs and routine data models. Screwing up an API contract is an expensive mistake. If you work at a large corporation you probably have lots of data models you can standardize on and API endpoints. If you’re starting from the ground up, it seems like an easy win standardizing the information you already pass around.