Signing Amazon Product Advertising API – C#/WCF – Part 2

Just the sample code

Download the sample project from:

AmazonProductAdvtApiWcfSample.zip, 324KB, 8/1/2009

The sample is self-contained, and does not require you to read this post to use. If trying to replicate the sample’s behavior in a different program, take a look at step 2 below regarding adding the Amazon ECS service reference.

Part 2

In part 1, we built a simple C# console application that used WCF to send an authenticated ItemSearch request to the Amazon Product Advertising API (formerly the Amazon Associates Web Service — AWS). If you are just starting out with authenticated requests to that service, you might want to read that post first.

While the sample worked well for some of you, others reported difficulties with more complicated usage patterns. I will try to tackle these issues in this post. I am also updating the sample solution to include additional sample programs.

Asynchronous requests

The sample program in part 1 showed how to send a synchronous request to the product advertising web service. The program prepared a request object, and then called an ItemSearch method. This method sent the request, and then waited for the reply to come back. In many situations, we do not want the program to wait. Instead, we want to accomplish other tasks until the response arrives.

If we could send an ItemSearch request asynchronously instead, the method would return immediately, allowing the program to continue doing other things. We would then expect a callback method to be invoked when the request completes, allowing us to make use of the results. Luckily, WCF supports asynchronous methods. Here’s what you could do:

Step 1 – Create a sample console application

  1. In Visual Studio 2008, select New/Project… from the File menu.
  2. Select Console Application from Visual C#/Windows, and enter the application’s name.
  3. Click OK

Visual Studio creates a new console application. So far, this isn’t different than any other console app.

Step 2 – Add a web service reference

  1. In the Solution Explorer view, right click References, then select Add Service Reference…
  2. The Add Service Reference dialog appears. Type in the Amazon Product Advertising API WSDL URL in the Address box:http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl
  3. Click Go. Visual Studio takes a moment to download the service description, and then populates the Services box. You should see AWSECommerceService in there.
  4. Type in the namespace for the code that Visual Studio will generate. I used Amazon.ECS.
  5. Click Advanced… Check the Generate asynchronous operations box. This will create the asynchronous versions of the methods.
  6. Click OK.

Visual Studio now creates a service reference and adds it to your project. You can see it under the Service References folder in Solution Explorer. Your application will now also have an app.config file added, listing WCF binding and endpoint information for the added service.

Step 3 – Add the helper classes that aid in signing the requests.

You can just copy and paste the Amazon.ECS.Addons folder from the sample project, with its three classes: AmazonHeader, AmazonSigningEndpointBehavior and AmazonSigningMessageInspector.

Step 4 – Add code to access the product advertising API asynchronously.

This code is modeled after the sample in part 1:

using System;
using System.ServiceModel;
using Async.Amazon.ECS;

namespace Async {
    class Program {
        // your Amazon ID's
        private const string accessKeyId = "XXXXXXXXXXXXXXXXXXXX";
        private const string secretKey   = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            BasicHttpBinding binding       = new BasicHttpBinding(BasicHttpSecurityMode.Transport);
            binding.MaxReceivedMessageSize = int.MaxValue;
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                binding,
                new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // add authentication to the ECS client
            client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior(accessKeyId, secretKey));

            // prepare an ItemSearch request
            ItemSearchRequest request = new ItemSearchRequest();
            request.SearchIndex       = "Books";
            request.Title             = "WCF";
            request.ResponseGroup     = new string[] { "Small" };

            ItemSearch itemSearch     = new ItemSearch();
            itemSearch.Request        = new ItemSearchRequest[] { request };
            itemSearch.AWSAccessKeyId = accessKeyId;

            // prepare to handle the results, once the async request is completed
            client.ItemSearchCompleted += (source, e) => {
                // write out the results
                foreach (var item in e.Result.Items[0].Item) {
                    Console.WriteLine(item.ItemAttributes.Title);
                }
            };

            // issue the ItemSearch request
            client.ItemSearchAsync(itemSearch);
            Console.WriteLine("Waiting for results...");

            // wait for results
            Console.ReadKey();
        }
    }
}

Like in part 1, this program creates a client object, associates it with an endpoint behavior that ensures requests will be authenticated, and creates an ItemSearch request object. This is where things change, however. While in part 1 our program just used the client to send the request and wait for the response. Our version is different.

First, we attach an event handler to the client’s ItemSearchCompleted event. The event handler will get called when the response to our request comes back. Our event handler here just prints out the item titles to the console. If you find the (source, e) => { … } syntax a bit strange — this is just a C# shorthand notation for defining a named event handler, perhaps something like void RequestCompleted(object sender, ItemSearchCompletedEventArgs e) { … }.

Only then does the program invoke the ItemSearch operation, by calling ItemSearchAsync. ItemSearchAsync sends the request to the web service, and returns immediately. At this point our program continues processing, by printing out that it’s waiting for results, and then waiting for a key press. This gives the asynchronous operation time to complete and invoke our event handler. A real application could use the opportunity to carry out real work here.

Note that the request is signed exactly the same way it was in part 1. There is no difference in how synchronous and asynchronous requests are authenticated. In fact, the SOAP messages look identical. The distinction is only in how these operations are used in your application.

Batch requests

The product advertising API allows you to combine two requests of the same type in one SOAP message. This practice is named batch requests. Batch requests are authenticated exactly the same way that single requests are. To try this out, follow step 1-3 above, then try out this main program:

using System;
using System.ServiceModel;
using Batch.Amazon.ECS;

namespace Batch {
    class Program {
        // your Amazon ID's
        private const string accessKeyId = "XXXXXXXXXXXXXXXXXXXX";
        private const string secretKey   = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            BasicHttpBinding binding       = new BasicHttpBinding(BasicHttpSecurityMode.Transport);
            binding.MaxReceivedMessageSize = int.MaxValue;
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                binding,
                new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // add authentication to the ECS client
            client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior(accessKeyId, secretKey));

            // prepare the first ItemSearchRequest
            ItemSearchRequest request1 = new ItemSearchRequest();
            request1.SearchIndex       = "Books";
            request1.Keywords          = "WCF";
            request1.ItemPage          = "1";
            request1.ResponseGroup     = new string[] { "Small" };

            // prepare a second ItemSearchRequest
            ItemSearchRequest request2  = new ItemSearchRequest();
            request2.SearchIndex        = "Books";
            request2.Keywords           = "WCF";
            request2.ItemPage           = "2";
            request2.ResponseGroup      = new string[] { "Small" };

            // batch the two requests together
            ItemSearch itemSearch       = new ItemSearch();
            itemSearch.Request          = new ItemSearchRequest[] { request1, request2 };
            itemSearch.AWSAccessKeyId   = accessKeyId;

            // issue the ItemSearch request
            ItemSearchResponse response = client.ItemSearch(itemSearch);

            // write out the results
            foreach (var item in response.Items[0].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
            foreach (var item in response.Items[1].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
        }
    }
}

This variation of the sample program creates two ItemSearchRequest objects instead of just one, and adds them as an array to an ItemSearch object before invoking the web service request, with:

itemSearch.Request = new ItemSearchRequest[] { request1, request2 };

When the response is returned, the program prints out the results from both requests separately.

Note how the requests obtain results for the first two pages of the same query in one go. Those could have been completely different queries too.

A simpler constructor for AmazonSigningEndpointBehavior

A recurring stumbling block for readers of part 1 was the constructor of the AmazonSigningEndpointBehavior class. The first constructor argument was the operation name — the string that is embedded in the SOAP request as the Action header. The constructor needed to track the operation name, in order to include it as part of the request signature. For an ItemSearch operation, this value was supposed to be “ItemSearch”. For an asynchronous ItemSearchAsync operation, this value was supposed to be “ItemSearch” too.

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Header xmlns:aws="http://security.amazonaws.com/doc/2007-01-01/" xmlns:a="http://schemas.microsoft.com/ws/2005/05/addressing/none">
    <a:Action s:mustUnderstand="1">
        http://soap.amazon.com/ItemSearch
    </a:Action>
    <aws:AWSAccessKeyId>1Y0XKFB8S7MD7GG73PR2</aws:AWSAccessKeyId>
    <aws:Timestamp>2009-08-13T00:27:59Z</aws:Timestamp>
    <aws:Signature>bm9FTq7HWnVgYusDaCbMyRrGTQXwA3Ji8FcdLEXAMPLE</aws:Signature>
  </s:Header>
// add authentication to the ECS client
client.ChannelFactory.Endpoint.Behaviors.Add(
    new AmazonSigningEndpointBehavior(
        "ItemSearch", accessKeyId, secretKey));

There was an opportunity for confusion here. Some folks placed ItemSearchAsync here, or BeginItemSearch, which wouldn’t work. To avoid the problem, the endpoint behavior constructor no longer requires an operation name. Instead, the operation is extracted from the Action header, just before the request is signed. If you are using the helper classes in the updated sample, you don’t need to do anything to take advantage of that.

// extract the operation from the url in the Action header,
// past the last '/' character.
string operation = Regex.Match(
    request.Headers.Action,
    "[^/]+$").ToString();

MaxReceivedMessageSize

Product advertising API responses can grow quite long, especially once you go beyond the puny “Small” response group. This quickly overwhelms the 16KB default limit in our WCF client code, causing an exception in client.ItemSearch(…). To avoid the exception, set the MaxReceivedMessageSize property of the binding object to some larger value. The sample program does this:

BasicHttpBinding binding = new BasicHttpBinding(
    BasicHttpSecurityMode.Transport);
binding.MaxReceivedMessageSize = int.MaxValue;

What’s in the sample code?

The sample solution has three console applications:

  1. Simple — just the bare-bones sample, issuing a synchronous ItemSearch request
  2. Batch — a modified version of Simple, sending two batched requests in one SOAP message
  3. Async — a modified version of Simple, sending an asynchronous request and waiting for the response

All three samples authenticate their requests, using the exact same three helper classes.

To run the samples, don’t forget to substitute your own Amazon access key and secret key, at the constants at the top of Program.cs. The samples do not include valid keys, and will not return any results from the Amazon web service without your valid keys.

More information

How to: Call WCF Service Operations Asynchronously (MSDN)

My thanks to Erica Sanders, Jason Foglia and Andrew for helping figure all this out.

Advertisements

Signing Amazon Product Advertising API requests — the missing C# WCF sample

(See also part 2 of this post)

Just the sample code

Download the sample project from:

AmazonProductAdvtApiWcfSample.zip, 324KB, 8/1/2009

The sample is self-contained. You don’t need to follow any of the text in this post to build and use it. If trying to replicate the sample’s behavior in a different program, take a look at step 2 below regarding adding the Amazon ECS service reference.

Authentication of Product Advertising API requests

If you develop for the Amazon Product Advertising API (formerly known as the Amazon Associates Web Service — AWS), you would have noticed by now the repeating emails from Amazon about a change in the rules. Starting August 15, 2009, all API requests must be authenticated using cryptographic signatures. The Amazon API documentation explains how to authenticate requests. The API team also provides sample code in C#, Java and Perl. Two C# samples are provided, one for applications using the SOAP interface, and another for applications using REST.

My application uses SOAP, and so I started with that sample. The SOAP sample, it turned out, uses the Microsoft.Web.Services3 namespace, which belongs to the Web Services Enhancements (WSE) class library. WSE appears to be a .NET 2.0, Visual Studio 2005 thing, and is superceded by WCF. If I have to touch my application, I would much rather update it to use WCF, .NET 3.5, and Visual Studio 2008 than to WSE.

Thankfully, a number of folksĀ tackled the problem already. The WCF solution implements theĀ IClientMessageInspector interface to gain an opportunity to sign SOAP requests just before they are sent. The signing logic isn’t particularly complex. It must, however, implement the letter of the specification,or requests are dropped. The remainder of this post outlines a short sample program illustrating how to generate authenticated WCF requests to the Amazon product advertising API. You can download the complete sample from the link at the top of this post.

Step 1 – Create a sample console application

  1. In Visual Studio 2008, select New/Project… from the File menu.
  2. Select Console Application from Visual C#/Windows, and enter the application’s name.
  3. Click OK

Visual Studio creates a new console application. So far, this isn’t different than any other console app.

Step 2 – Add a web service reference

  1. In the Solution Explorer view, right click References, then select Add Service Reference…
  2. The Add Service Reference dialog appears. Type in the Amazon Product Advertising API WSDL URL in the Address box: http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl.
  3. Click Go. Visual Studio takes a moment to download the service description, and then populates the Services box. You should see AWSECommerceService in there.
  4. Type in the namespace for the code that Visual Studio will generate. I used Amazon.ECS. Click OK.

Visual Studio now creates a service reference and adds it to your project. You can see it under the Service References folder in Solution Explorer. Your application will now also have an app.config file added, listing WCF binding and endpoint information for the added service.

Step 3 – Add code to access the Product Advertising API.

You could update your program as follows to have it connect to the product advertising API and search for books with “WCF” in their title. Make sure to add a using statement to reference the namespace you chose for the service in step 2.

    using System.ServiceModel;
    using AmazonProductAdvtApiWcfSample.Amazon.ECS;

    class Program {
        // your Amazon ID's
        private const string accessKeyId    = "YOURACCESSKEYIDHEREX";
        private const string secretKey      = "YOURAMAZONSECRETKEYHERE/YESITSTHISLONGXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                new BasicHttpBinding(),
                new EndpointAddress("http://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // prepare an ItemSearch request
            ItemSearchRequest request   = new ItemSearchRequest();
            request.SearchIndex         = "Books";
            request.Title               = "WCF";
            request.ResponseGroup       = new string[] { "Small" };

            ItemSearch itemSearch       = new ItemSearch();
            itemSearch.Request          = new ItemSearchRequest[] { request };
            itemSearch.AWSAccessKeyId   = accessKeyId;

            // issue the ItemSearch request
            ItemSearchResponse response = client.ItemSearch(itemSearch);

            // write out the results
            foreach (var item in response.Items[0].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
        }
    }

If you enter your Amazon access key ID into accessKeyId and run this program before the August 15 deadline, you might just obtain results. Your console should list the top 10 results for the “WCF” query against Amazon’s US book catalog. If you read this after the deadline has expired, you must complete the remaining steps before you can see the results.

Step 4 — Add the authentication code

That’s why we’re here. This sample adds the authentication SOAP headers using three classes:

  • AmazonSigningMessageInspector implements the IClientMessageInspector interface. The implementation of the BeforeSendRequest method in this class generates the request signature and adds the appropriate headers to the request.
  • AmazonSigningEndpointBehavior implements the IEndpointBehavior. The implementation of the ApplyClientBehavior in this class adds our message inspector to the ECS client at the right moment.
  • AmazonHeader is a small helper class implementing MessageHeader. Our message inspector uses instances of this class to add the authentication headers into the SOAP request.

Rather than include the full listing of these classes, let’s focus on the portion that actually signs the request.

public object BeforeSendRequest(ref Message request, IClientChannel channel) {
    // prepare the data to sign
    DateTime    now             = DateTime.UtcNow;
    string      timestamp       = now.ToString("yyyy-MM-ddTHH:mm:ssZ");
    string      signMe          = operation + timestamp;
    byte[]      bytesToSign     = Encoding.UTF8.GetBytes(signMe);

    // sign the data
    byte[]      secretKeyBytes  = Encoding.UTF8.GetBytes(secretKey);
    HMAC        hmacSha256      = new HMACSHA256(secretKeyBytes);
    byte[]      hashBytes       = hmacSha256.ComputeHash(bytesToSign);
    string      signature       = Convert.ToBase64String(hashBytes);

    // add the signature information to the request headers
    request.Headers.Add(new AmazonHeader("AWSAccessKeyId", accessKeyId));
    request.Headers.Add(new AmazonHeader("Timestamp", timestamp));
    request.Headers.Add(new AmazonHeader("Signature", signature));

    return null;
}

BeforeSendRequest gets called just before the SOAP request is put out on the network. Our implementation of this method comutes a hash-based message authentication code (HMAC) of the operation and current time using the SHA256 hash function. It also puts the access key ID, timestamp and signature in the SOAP request headers. This embeds tags like the following in the SOAP request:

<soap:Header
   xmlns:aws="http://security.amazonaws.com/doc/2007-01-01/">
   <aws:AWSAccessKeyId>YOURACCESSKEYIDHEREX</aws:AWSAccessKeyId>
   <aws:Timestamp>2009-08-15T23:59:59Z</aws:Timestamp>
   <aws:Signature>SZf1CHmQnrZbsrC13hCZS061ywsEXAMPLE</aws:Signature>
</soap:Header>

Step 5 – Make the Amazon ECS client authenticate

Finally, to make our Amazon ECS client actually authenticate its requests, modify the main() method as follows:

// create a WCF Amazon ECS client
AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
    new BasicHttpBinding(BasicHttpSecurityMode.Transport),
    new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

// add authentication to the ECS client
client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior("ItemSearch", accessKeyId, secretKey));

There are three changes from our original console program:

  1. The binding constructor now takes a BasicHttpSecurityMode.Transport argument. This forces an HTTPS connection. The Amazon web service refuses to accept authenticated requests over plain HTTP.
  2. The endpoint address starts with “https:” instead of “http:”. The remainder of the URL did not change.
  3. An instance of our endpoint behavior class is added to the client’s channel factory. This ensures that our message inspector will get plugged in when the client creates new connections, and will get to sign outgoing SOAP messages.

That’s it.

This is all that’s required to get your WCF based Amazon Product Advertising API application to send authenticated requests. If you build and run the application now, your console should list 10 book titles returned from the service.

Worked for you? Didn’t work? Let us know in the comments!

Read part 2.