Some sample tags

The Tag manager website is great for creating a handful of tags. If you have to create more of them, however, Tag manager can become tiring, and fast. To that end, Microsoft provides the Tag API. The Tag API lets developers create and update several tags at once. The Tag API is available as a SOAP web service that can be invoked from numerous software environments. In this article, we will put together a small .NET Framework applet that lets you create a any number of tags.

Getting started with the Tag API

Visit http://tag.microsoft.com/developer/index.aspx to get acquainted with the Tag API. To get started, you will need to set up your free Tag Manager account, and then obtain an API key. If you haven’t already, go to the tag manager website and click the TAG MANAGER link at the top right to get started. The signup process is straightforward. Once inside the Tag manager, you can see your (empty) list of tags. At this point, you should be able to request an API key. The API key looks like a string of 30 or so hex digits, not unlike a GUID.

Your first TagAPI program

With the API key in hand, we can get started. Fire up Visual Studio, and create a new console application. In the Solution Explorer view, right click References and select Add Service Reference… In the Add Service Reference dialog window that pops up, type in the address of the Tag API: https://ws.tag.microsoft.com/MIBPService.wsdl and click Go. The MIBPService will show up in the Services list. If you expand it and select the IMIBPContract underneath, the list of supported operations will show. Modify the namespace  to “Tag” and click OK to proceed. A service reference to the Tag API will be added to your project.

Adding a service reference

Adding the Tag API service reference

To use the Tag API in your program, first add a using statement with its namespace. I named my application “MakeTags” — use your application’s namespace as the prefix there. The second part of the namespace, “Tag”, is the value entered in the Namespace box of the Add Service Reference window.

using namespace MakeTags.Tag;

To invoke Tag API services, we will need to create two objects. One represents the Tag API service, and the other represents our API key. Make sure to insert the API key you applied for into creds.AccessToken. In your program’s Main function, add:

static void Main(string[] args)
{
    MIBPContractClient tagService = new MIBPContractClient();
    UserCredential creds = new UserCredential();
    creds.AccessToken = "your-access-token-here";
}

To create a single Tag, we could add the following code to invoke the CreateTag method. Make sure to use a new tag title. Trying to create a tag with a title of a tag you created before will fail. Also note that the tag category has to exist before you call CreateTag. I used the “Main” category, that is supposed to be created for you on your new Tag account. But it has been a long time since I created my account, and I may be wrong. If needed, you can create a category from Tag Manager, or invoke the CreateCategory method to create a new one.

string tagTitle = "Tag title here";
string category = "Main";

URITag tag = new URITag();
tag.Title = tagTitle;
tag.MedFiUrl = "http://flyingpies.wordpress.com/2011/05/24/creating-several-microsoft-tags";
tag.UTCStartDate = DateTime.UtcNow;
tagService.CreateTag(creds, category, tag);

At this point, if all is well, the program can run and create a new tag. Try it out, and look for your new tag inside Tag Manager. Running the unmodified program again will fail — because we already have a tag with this name. Modify the tag name before you run the program one more time. Here’s the complete program:

using System;
using MakeTags.Tag;

namespace MakeTags {
    class Program {
        static void Main(string[] args) {
            MIBPContractClient tagService = new MIBPContractClient();
            UserCredential creds = new UserCredential();
            creds.AccessToken = "your-access-token-here";

            string tagTitle = "Tag title here";
            string category = "Main";

            URITag tag = new URITag();
            tag.Title = tagTitle;
            tag.MedFiUrl = "http://flyingpies.wordpress.com/2011/05/24/creating-several-microsoft-tags";
            tag.UTCStartDate = DateTime.UtcNow;
            tagService.CreateTag(creds, category, tag);
        }
    }
}

Rendering the tags

To print the tags, we will need to render them as images. The GetBardcode method asks the Tag API service to render the tag for us in one of several image formats. In addition to several bitmap formats (png, jpeg, gif, tiff), GetBarcode can also return a PDF rendering of the tag. You can also specify whether you want the rendered tag to include instructions for the user to download the tag mobile phone app.

Adding the following at the bottom of the program almost does what you would expect:

string tagImageFilePath = "mytag.png";
byte[] tagImageBytes = tagService.GetBarcode(
    creds,
    category,
    tagTitle,
    ImageTypes.png,
    1f,
    DecorationType.HCCBRP_DECORATION_DOWNLOAD,
    false);
System.IO.File.WriteAllBytes(tagImageFilePath, tagImageBytes);

If you run the program at this state (make sure to use a new title, or delete the old tags), it will fail in GetBarcode. The exception complains: “…The maximum array length quota (16384) has been exceeded while reading XML data. This quota may be increased by changing the MaxArrayLength property on the…”. The service reference that we added in the beginning limited itself to 16K of data, but our tag image apparently needs a few more bytes (at our chosen image format and size, about 45K are needed.) This is easy to fix. In the application’s app.config file, find the line that contains maxArrayLength=”16384″ and replace it with a higher value. I used maxArrayLength=”100000″:

<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="100000"
    maxBytesPerRead="4096" maxNameTableCharCount="16384" />

Now our program runs to completion, and leaves an image file of the rendered tag in its startup folder. Here’s the image created from the snippet above. Go ahead, scan it with the tag reader on your mobile phone!

A sample rendered tag

Several at once

Now that we know how to create and render tags, we can do so in a loop. Here’s the complete program:

using System;
using System.IO;
using MakeTags.Tag;

namespace MakeTags {
    class Program {
        static void Main(string[] args) {
            MIBPContractClient tagService = new MIBPContractClient();
            UserCredential creds = new UserCredential();
            creds.AccessToken = "your-access-token-here";

            int tagsToCreate = 10;
            string category = "Main";
            string tagTitlePrefix = "My Sample Tag ";
            string tagImageFilePathFormat = "mytag{0}.png";

            for (int i = 0; i < tagsToCreate; ++i) {
                Console.WriteLine("Creating tag " + i);

                string tagTitle = tagTitlePrefix + i;

                URITag tag = new URITag();
                tag.Title = tagTitle;
                tag.MedFiUrl = "http://flyingpies.wordpress.com/2011/05/24/creating-several-microsoft-tags";
                tag.UTCStartDate = DateTime.UtcNow;
                tagService.CreateTag(creds, category, tag);

                string tagImageFilePath = string.Format(tagImageFilePathFormat, i);
                byte[] tagImageBytes = tagService.GetBarcode(
                    creds,
                    category,
                    tagTitle,
                    ImageTypes.png,
                    1f,
                    DecorationType.HCCBRP_DECORATION_DOWNLOAD,
                    false);
                File.WriteAllBytes(tagImageFilePath, tagImageBytes);
            }
        }
    }
}

The rendered tags

Measures a raw value: “shows the most recently observed value.”

using System.Diagnostics;
using System.Threading;

namespace PerformanceCounterExperiments
{
    class Program
    {
        static void Main(string[] args)
        {
            string categoryName = "Experimental";
            string counterName = "number of items";

            // delete the category if it already exists
            if (PerformanceCounterCategory.Exists(categoryName))
            {
                PerformanceCounterCategory.Delete(categoryName);
            }

            // create the performance category and counter
            PerformanceCounterCategory.Create(
                categoryName,
                "",
                PerformanceCounterCategoryType.SingleInstance,
                new CounterCreationDataCollection(new[]
                {
                    new CounterCreationData(
                        counterName,
                        "",
                        PerformanceCounterType.NumberOfItems32)
                }));

            // obtain the writable counter object
            PerformanceCounter counter = new PerformanceCounter(
                categoryName,
                counterName,
                false);

            // increment the counter every 100 milliseconds
            Console.WriteLine("Starting");
            while (true)
            {
                counter.Increment();
                Thread.Sleep(100);
            }
        }
    }
}

AverageCount64

Measures ITEMS per OPERATION: ”how many items are processed, on average, during an operation.”

// create the performance category and counters
PerformanceCounterCategory.Create(
    categoryName,
    "",
    PerformanceCounterCategoryType.SingleInstance,
    new CounterCreationDataCollection(new[]
    {
        new CounterCreationData(
            counterName,
            "",
            PerformanceCounterType.AverageCount64),
        new CounterCreationData(
            counterName + " base",
            "",
            PerformanceCounterType.AverageBase)
    }));

// obtain the writable counter objects
PerformanceCounter counter = new PerformanceCounter(
    categoryName,
    counterName,
    false);
PerformanceCounter baseCounter = new PerformanceCounter(
    categoryName,
    counterName + " base",
    false);

// increment the counter every 100 milliseconds
Console.WriteLine("Starting");
Random random = new Random();

while (true)
{
    counter.IncrementBy(50 + random.Next(-10, 10));
    baseCounter.Increment();
    Thread.Sleep(100);
}

AverageTimer32

Measures TIME, in seconds: ”the time it takes, on average, to complete a process or operation.”

This chart shows average operation time, in milliseconds. The counter actually measures the time in seconds. We multiplied it by x1000 in Performance Monitor, so that the 50 milliseconds value would show as 50.

// create the performance category and counters
PerformanceCounterCategory.Create(
    categoryName,
    "",
    PerformanceCounterCategoryType.SingleInstance,
    new CounterCreationDataCollection(new[]
    {
        new CounterCreationData(
            counterName,
            "",
            PerformanceCounterType.AverageTimer32),
        new CounterCreationData(
            counterName + " base",
            "",
            PerformanceCounterType.AverageBase)
    }));

// obtain the writable counter objects
PerformanceCounter counter = new PerformanceCounter(
    categoryName,
    counterName,
    false);
PerformanceCounter baseCounter = new PerformanceCounter(
    categoryName,
    counterName + " base",
    false);

// do a 50 msec operation every 100 msec, and measure the time
Console.WriteLine("Starting");
Random random = new Random();

while (true)
{
    // simulate an operation taking ~50 msec
    Stopwatch stopwatch = Stopwatch.StartNew();
    Thread.Sleep(50 + random.Next(-10, 10));
    stopwatch.Stop();

    // update the counter and base with the number of ticks
    // the operation took
    counter.IncrementBy(stopwatch.ElapsedTicks);
    baseCounter.Increment();

    Thread.Sleep(100);
}

CountPerTimeInterval32

Measures the TIME PERCENTAGE an operation takes out of a larger INTERVAL. The MSDN documentation appears to be somewhat misleading as to this counter type.

// create the performance category and counters
PerformanceCounterCategory.Create(
    categoryName,
    "",
    PerformanceCounterCategoryType.SingleInstance,
    new CounterCreationDataCollection(new[]
    {
        new CounterCreationData(
            counterName,
            "",
            PerformanceCounterType.CountPerTimeInterval32)
    }));

// obtain the writable counter objects
PerformanceCounter counter = new PerformanceCounter(
    categoryName,
    counterName,
    false);

// do a 100 msec operation every 250 msec, and measure the time
Console.WriteLine("Starting");
Random random = new Random();

while (true)
{
    Stopwatch stopwatch = Stopwatch.StartNew();
    Thread.Sleep(100);
    stopwatch.Stop();

    counter.IncrementBy(stopwatch.ElapsedTicks);

    Thread.Sleep(150);
}

RateOfCountsPerSecond32

Measures OPERATIONS per SECOND: “shows the average number of operations completed during each second.”

// create the performance category and counters
PerformanceCounterCategory.Create(
    categoryName,
    "",
    PerformanceCounterCategoryType.SingleInstance,
    new CounterCreationDataCollection(new[]
    {
        new CounterCreationData(
            counterName,
            "",
            PerformanceCounterType.RateOfCountsPerSecond32)
    }));

// obtain the writable counter objects
PerformanceCounter counter = new PerformanceCounter(
    categoryName,
    counterName,
    false);

// increment the counter every 100 milliseconds
Console.WriteLine("Starting");
Random random = new Random();

while (true)
{
    counter.IncrementBy(5);

    Thread.Sleep(100);
}

Reference

http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecountertype.aspx

Download the sample project from:

AmazonProductAdvtApiWcfSample.zip, 324KB, 8/1/2009

The sample is self-contained, and does not require you to read this post to use. If trying to replicate the sample’s behavior in a different program, take a look at step 2 below regarding adding the Amazon ECS service reference.

Part 2

In part 1, we built a simple C# console application that used WCF to send an authenticated ItemSearch request to the Amazon Product Advertising API (formerly the Amazon Associates Web Service — AWS). If you are just starting out with authenticated requests to that service, you might want to read that post first.

While the sample worked well for some of you, others reported difficulties with more complicated usage patterns. I will try to tackle these issues in this post. I am also updating the sample solution to include additional sample programs.

Asynchronous requests

The sample program in part 1 showed how to send a synchronous request to the product advertising web service. The program prepared a request object, and then called an ItemSearch method. This method sent the request, and then waited for the reply to come back. In many situations, we do not want the program to wait. Instead, we want to accomplish other tasks until the response arrives.

If we could send an ItemSearch request asynchronously instead, the method would return immediately, allowing the program to continue doing other things. We would then expect a callback method to be invoked when the request completes, allowing us to make use of the results. Luckily, WCF supports asynchronous methods. Here’s what you could do:

Step 1 – Create a sample console application

1. In Visual Studio 2008, select New/Project… from the File menu.
2. Select Console Application from Visual C#/Windows, and enter the application’s name.
3. Click OK

Visual Studio creates a new console application. So far, this isn’t different than any other console app.

Step 2 – Add a web service reference

1. In the Solution Explorer view, right click References, then select Add Service Reference…
2. The Add Service Reference dialog appears. Type in the Amazon Product Advertising API WSDL URL in the Address box:http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl
3. Click Go. Visual Studio takes a moment to download the service description, and then populates the Services box. You should see AWSECommerceService in there.
4. Type in the namespace for the code that Visual Studio will generate. I used Amazon.ECS.
5. Click Advanced… Check the Generate asynchronous operations box. This will create the asynchronous versions of the methods.
6. Click OK.

Visual Studio now creates a service reference and adds it to your project. You can see it under the Service References folder in Solution Explorer. Your application will now also have an app.config file added, listing WCF binding and endpoint information for the added service.

Step 3 – Add the helper classes that aid in signing the requests.

You can just copy and paste the Amazon.ECS.Addons folder from the sample project, with its three classes: AmazonHeader, AmazonSigningEndpointBehavior and AmazonSigningMessageInspector.

Step 4 – Add code to access the product advertising API asynchronously.

This code is modeled after the sample in part 1:

using System;
using System.ServiceModel;
using Async.Amazon.ECS;

namespace Async {
    class Program {
        // your Amazon ID's
        private const string accessKeyId = "XXXXXXXXXXXXXXXXXXXX";
        private const string secretKey   = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            BasicHttpBinding binding       = new BasicHttpBinding(BasicHttpSecurityMode.Transport);
            binding.MaxReceivedMessageSize = int.MaxValue;
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                binding,
                new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // add authentication to the ECS client
            client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior(accessKeyId, secretKey));

            // prepare an ItemSearch request
            ItemSearchRequest request = new ItemSearchRequest();
            request.SearchIndex       = "Books";
            request.Title             = "WCF";
            request.ResponseGroup     = new string[] { "Small" };

            ItemSearch itemSearch     = new ItemSearch();
            itemSearch.Request        = new ItemSearchRequest[] { request };
            itemSearch.AWSAccessKeyId = accessKeyId;

            // prepare to handle the results, once the async request is completed
            client.ItemSearchCompleted += (source, e) => {
                // write out the results
                foreach (var item in e.Result.Items[0].Item) {
                    Console.WriteLine(item.ItemAttributes.Title);
                }
            };

            // issue the ItemSearch request
            client.ItemSearchAsync(itemSearch);
            Console.WriteLine("Waiting for results...");

            // wait for results
            Console.ReadKey();
        }
    }
}

Like in part 1, this program creates a client object, associates it with an endpoint behavior that ensures requests will be authenticated, and creates an ItemSearch request object. This is where things change, however. While in part 1 our program just used the client to send the request and wait for the response. Our version is different.

First, we attach an event handler to the client’s ItemSearchCompleted event. The event handler will get called when the response to our request comes back. Our event handler here just prints out the item titles to the console. If you find the (source, e) => { … } syntax a bit strange — this is just a C# shorthand notation for defining a named event handler, perhaps something like void RequestCompleted(object sender, ItemSearchCompletedEventArgs e) { … }.

Only then does the program invoke the ItemSearch operation, by calling ItemSearchAsync. ItemSearchAsync sends the request to the web service, and returns immediately. At this point our program continues processing, by printing out that it’s waiting for results, and then waiting for a key press. This gives the asynchronous operation time to complete and invoke our event handler. A real application could use the opportunity to carry out real work here.

Note that the request is signed exactly the same way it was in part 1. There is no difference in how synchronous and asynchronous requests are authenticated. In fact, the SOAP messages look identical. The distinction is only in how these operations are used in your application.

Batch requests

The product advertising API allows you to combine two requests of the same type in one SOAP message. This practice is named batch requests. Batch requests are authenticated exactly the same way that single requests are. To try this out, follow step 1-3 above, then try out this main program:

using System;
using System.ServiceModel;
using Batch.Amazon.ECS;

namespace Batch {
    class Program {
        // your Amazon ID's
        private const string accessKeyId = "XXXXXXXXXXXXXXXXXXXX";
        private const string secretKey   = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            BasicHttpBinding binding       = new BasicHttpBinding(BasicHttpSecurityMode.Transport);
            binding.MaxReceivedMessageSize = int.MaxValue;
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                binding,
                new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // add authentication to the ECS client
            client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior(accessKeyId, secretKey));

            // prepare the first ItemSearchRequest
            ItemSearchRequest request1 = new ItemSearchRequest();
            request1.SearchIndex       = "Books";
            request1.Keywords          = "WCF";
            request1.ItemPage          = "1";
            request1.ResponseGroup     = new string[] { "Small" };

            // prepare a second ItemSearchRequest
            ItemSearchRequest request2  = new ItemSearchRequest();
            request2.SearchIndex        = "Books";
            request2.Keywords           = "WCF";
            request2.ItemPage           = "2";
            request2.ResponseGroup      = new string[] { "Small" };

            // batch the two requests together
            ItemSearch itemSearch       = new ItemSearch();
            itemSearch.Request          = new ItemSearchRequest[] { request1, request2 };
            itemSearch.AWSAccessKeyId   = accessKeyId;

            // issue the ItemSearch request
            ItemSearchResponse response = client.ItemSearch(itemSearch);

            // write out the results
            foreach (var item in response.Items[0].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
            foreach (var item in response.Items[1].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
        }
    }
}

This variation of the sample program creates two ItemSearchRequest objects instead of just one, and adds them as an array to an ItemSearch object before invoking the web service request, with:

itemSearch.Request = new ItemSearchRequest[] { request1, request2 };

When the response is returned, the program prints out the results from both requests separately.

Note how the requests obtain results for the first two pages of the same query in one go. Those could have been completely different queries too.

A simpler constructor for AmazonSigningEndpointBehavior

A recurring stumbling block for readers of part 1 was the constructor of the AmazonSigningEndpointBehavior class. The first constructor argument was the operation name — the string that is embedded in the SOAP request as the Action header. The constructor needed to track the operation name, in order to include it as part of the request signature. For an ItemSearch operation, this value was supposed to be “ItemSearch”. For an asynchronous ItemSearchAsync operation, this value was supposed to be “ItemSearch” too.

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Header xmlns:aws="http://security.amazonaws.com/doc/2007-01-01/" xmlns:a="http://schemas.microsoft.com/ws/2005/05/addressing/none">
    <a:Action s:mustUnderstand="1">
        http://soap.amazon.com/ItemSearch
    </a:Action>
    <aws:AWSAccessKeyId>1Y0XKFB8S7MD7GG73PR2</aws:AWSAccessKeyId>
    <aws:Timestamp>2009-08-13T00:27:59Z</aws:Timestamp>
    <aws:Signature>bm9FTq7HWnVgYusDaCbMyRrGTQXwA3Ji8FcdLEXAMPLE</aws:Signature>
  </s:Header>

// add authentication to the ECS client
client.ChannelFactory.Endpoint.Behaviors.Add(
    new AmazonSigningEndpointBehavior(
        "ItemSearch", accessKeyId, secretKey));

There was an opportunity for confusion here. Some folks placed ItemSearchAsync here, or BeginItemSearch, which wouldn’t work. To avoid the problem, the endpoint behavior constructor no longer requires an operation name. Instead, the operation is extracted from the Action header, just before the request is signed. If you are using the helper classes in the updated sample, you don’t need to do anything to take advantage of that.

// extract the operation from the url in the Action header,
// past the last '/' character.
string operation = Regex.Match(
    request.Headers.Action,
    "[^/]+$").ToString();

MaxReceivedMessageSize

Product advertising API responses can grow quite long, especially once you go beyond the puny “Small” response group. This quickly overwhelms the 16KB default limit in our WCF client code, causing an exception in client.ItemSearch(…). To avoid the exception, set the MaxReceivedMessageSize property of the binding object to some larger value. The sample program does this:

BasicHttpBinding binding = new BasicHttpBinding(
    BasicHttpSecurityMode.Transport);
binding.MaxReceivedMessageSize = int.MaxValue;

What’s in the sample code?

The sample solution has three console applications:

1. Simple — just the bare-bones sample, issuing a synchronous ItemSearch request
2. Batch — a modified version of Simple, sending two batched requests in one SOAP message
3. Async — a modified version of Simple, sending an asynchronous request and waiting for the response

All three samples authenticate their requests, using the exact same three helper classes.

To run the samples, don’t forget to substitute your own Amazon access key and secret key, at the constants at the top of Program.cs. The samples do not include valid keys, and will not return any results from the Amazon web service without your valid keys.

More information

How to: Call WCF Service Operations Asynchronously (MSDN)

My thanks to Erica Sanders, Jason Foglia and Andrew for helping figure all this out.

Download the sample project from:

AmazonS3SoapWcfSample.zip, 30KB, 8/4/2009

The sample is self-contained. You don’t need to follow any of the text in this post to build and use it. If trying to replicate the sample’s behavior in a different program, take a look at step 2 below regarding adding the Amazon S3 service reference

The shortest ever S3 C#/SOAP/WCF client

I put this together as an answer to a question on stackoverflow. What’s the simplest, most straight-forward way to access the Amazon S3 service from a C# program? To me, the obstacle has always been with authenticating the requests. You have to dig it out of Amazon’s sample code set up in the days before WCF. So here’s the shortest I could get it to:

1. Step 1: Start Visual Studio 2008, create a new C# Windows console application.

2. Step 2: Add the S3 WSDL as a service reference. In Solution Explorer, right click References, select Add Service Reference. Type in the S3 WSDL address in the Address box: http://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl. Click Go. “AmazonS3″ should show in the Services box. Enter a namespace. I entered Amazon.S3. Click OK.

3. Step 3: Modify Program.cs to look something like the following:

using System;
using System.Globalization;
using System.Text;
using System.Security.Cryptography;
using AmazonS3SoapWcfSample.Amazon.S3;

namespace AmazonS3SoapWcfSample {
	class Program {
		private const string accessKeyId     = "YOURACCESSKEYIDHERE0";
		private const string secretAccessKey = "YOURSECRETACCESSKEYHEREANDYESITSTHATLONG";

		public static DateTime LocalNow() {
			DateTime now = DateTime.Now;
			return new DateTime(
				now.Year, now.Month, now.Day,
				now.Hour, now.Minute, now.Second,
				now.Millisecond, DateTimeKind.Local);
		}

		public static string SignRequest(string secret, string operation, DateTime timestamp) {
			HMACSHA1 hmac         = new HMACSHA1(Encoding.UTF8.GetBytes(secret));
			string   isoTimeStamp = timestamp.ToUniversalTime().ToString(
				"yyyy-MM-ddTHH:mm:ss.fffZ",
				CultureInfo.InvariantCulture);
			string   signMe       = "AmazonS3" + operation + isoTimeStamp;
			string   signature    = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(signMe)));
			return signature;
		}

		static void Main(string[] args) {
			DateTime       now    = LocalNow();
			AmazonS3Client client = new AmazonS3Client();

			var result = client.ListAllMyBuckets(
				accessKeyId,
				now,
				SignRequest(secretAccessKey, "ListAllMyBuckets", now));

			foreach (var bucket in result.Buckets) {
				Console.WriteLine(bucket.Name);
			}
		}
	}
}

If you now insert your access key ID and secret access key in the appropriate spots and run the program, you should get a listing of your S3 buckets. The AmazonS3Client class has all the SOAP operations available as instance methods on it.

Works for you? Have an idea how to make it even shorter? –leave a comment.

Just the sample code

Download the sample project from:

AmazonProductAdvtApiWcfSample.zip, 324KB, 8/1/2009

The sample is self-contained. You don’t need to follow any of the text in this post to build and use it. If trying to replicate the sample’s behavior in a different program, take a look at step 2 below regarding adding the Amazon ECS service reference.

Authentication of Product Advertising API requests

If you develop for the Amazon Product Advertising API (formerly known as the Amazon Associates Web Service — AWS), you would have noticed by now the repeating emails from Amazon about a change in the rules. Starting August 15, 2009, all API requests must be authenticated using cryptographic signatures. The Amazon API documentation explains how to authenticate requests. The API team also provides sample code in C#, Java and Perl. Two C# samples are provided, one for applications using the SOAP interface, and another for applications using REST.

My application uses SOAP, and so I started with that sample. The SOAP sample, it turned out, uses the Microsoft.Web.Services3 namespace, which belongs to the Web Services Enhancements (WSE) class library. WSE appears to be a .NET 2.0, Visual Studio 2005 thing, and is superceded by WCF. If I have to touch my application, I would much rather update it to use WCF, .NET 3.5, and Visual Studio 2008 than to WSE.

Thankfully, a number of folks tackled the problem already. The WCF solution implements the IClientMessageInspector interface to gain an opportunity to sign SOAP requests just before they are sent. The signing logic isn’t particularly complex. It must, however, implement the letter of the specification,or requests are dropped. The remainder of this post outlines a short sample program illustrating how to generate authenticated WCF requests to the Amazon product advertising API. You can download the complete sample from the link at the top of this post.

Step 1 – Create a sample console application

1. In Visual Studio 2008, select New/Project… from the File menu.
2. Select Console Application from Visual C#/Windows, and enter the application’s name.
3. Click OK

Visual Studio creates a new console application. So far, this isn’t different than any other console app.

Step 2 – Add a web service reference

1. In the Solution Explorer view, right click References, then select Add Service Reference…
2. The Add Service Reference dialog appears. Type in the Amazon Product Advertising API WSDL URL in the Address box: http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl.
3. Click Go. Visual Studio takes a moment to download the service description, and then populates the Services box. You should see AWSECommerceService in there.
4. Type in the namespace for the code that Visual Studio will generate. I used Amazon.ECS. Click OK.

Visual Studio now creates a service reference and adds it to your project. You can see it under the Service References folder in Solution Explorer. Your application will now also have an app.config file added, listing WCF binding and endpoint information for the added service.

Step 3 – Add code to access the Product Advertising API.

You could update your program as follows to have it connect to the product advertising API and search for books with “WCF” in their title. Make sure to add a using statement to reference the namespace you chose for the service in step 2.

    using System.ServiceModel;
    using AmazonProductAdvtApiWcfSample.Amazon.ECS;

    class Program {
        // your Amazon ID's
        private const string accessKeyId    = "YOURACCESSKEYIDHEREX";
        private const string secretKey      = "YOURAMAZONSECRETKEYHERE/YESITSTHISLONGXX";

        // the program starts here
        static void Main(string[] args) {

            // create a WCF Amazon ECS client
            AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
                new BasicHttpBinding(),
                new EndpointAddress("http://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

            // prepare an ItemSearch request
            ItemSearchRequest request   = new ItemSearchRequest();
            request.SearchIndex         = "Books";
            request.Title               = "WCF";
            request.ResponseGroup       = new string[] { "Small" };

            ItemSearch itemSearch       = new ItemSearch();
            itemSearch.Request          = new ItemSearchRequest[] { request };
            itemSearch.AWSAccessKeyId   = accessKeyId;

            // issue the ItemSearch request
            ItemSearchResponse response = client.ItemSearch(itemSearch);

            // write out the results
            foreach (var item in response.Items[0].Item) {
                Console.WriteLine(item.ItemAttributes.Title);
            }
        }
    }

If you enter your Amazon access key ID into accessKeyId and run this program before the August 15 deadline, you might just obtain results. Your console should list the top 10 results for the “WCF” query against Amazon’s US book catalog. If you read this after the deadline has expired, you must complete the remaining steps before you can see the results.

Step 4 — Add the authentication code

That’s why we’re here. This sample adds the authentication SOAP headers using three classes:

• AmazonSigningMessageInspector implements the IClientMessageInspector interface. The implementation of the BeforeSendRequest method in this class generates the request signature and adds the appropriate headers to the request.
• AmazonSigningEndpointBehavior implements the IEndpointBehavior. The implementation of the ApplyClientBehavior in this class adds our message inspector to the ECS client at the right moment.
• AmazonHeader is a small helper class implementing MessageHeader. Our message inspector uses instances of this class to add the authentication headers into the SOAP request.

Rather than include the full listing of these classes, let’s focus on the portion that actually signs the request.

public object BeforeSendRequest(ref Message request, IClientChannel channel) {
    // prepare the data to sign
    DateTime    now             = DateTime.UtcNow;
    string      timestamp       = now.ToString("yyyy-MM-ddTHH:mm:ssZ");
    string      signMe          = operation + timestamp;
    byte[]      bytesToSign     = Encoding.UTF8.GetBytes(signMe);

    // sign the data
    byte[]      secretKeyBytes  = Encoding.UTF8.GetBytes(secretKey);
    HMAC        hmacSha256      = new HMACSHA256(secretKeyBytes);
    byte[]      hashBytes       = hmacSha256.ComputeHash(bytesToSign);
    string      signature       = Convert.ToBase64String(hashBytes);

    // add the signature information to the request headers
    request.Headers.Add(new AmazonHeader("AWSAccessKeyId", accessKeyId));
    request.Headers.Add(new AmazonHeader("Timestamp", timestamp));
    request.Headers.Add(new AmazonHeader("Signature", signature));

    return null;
}

BeforeSendRequest gets called just before the SOAP request is put out on the network. Our implementation of this method comutes a hash-based message authentication code (HMAC) of the operation and current time using the SHA256 hash function. It also puts the access key ID, timestamp and signature in the SOAP request headers. This embeds tags like the following in the SOAP request:

<soap:Header
   xmlns:aws="http://security.amazonaws.com/doc/2007-01-01/">
   <aws:AWSAccessKeyId>YOURACCESSKEYIDHEREX</aws:AWSAccessKeyId>
   <aws:Timestamp>2009-08-15T23:59:59Z</aws:Timestamp>
   <aws:Signature>SZf1CHmQnrZbsrC13hCZS061ywsEXAMPLE</aws:Signature>
</soap:Header>

Step 5 – Make the Amazon ECS client authenticate

Finally, to make our Amazon ECS client actually authenticate its requests, modify the main() method as follows:

// create a WCF Amazon ECS client
AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient(
    new BasicHttpBinding(BasicHttpSecurityMode.Transport),
    new EndpointAddress("https://webservices.amazon.com/onca/soap?Service=AWSECommerceService"));

// add authentication to the ECS client
client.ChannelFactory.Endpoint.Behaviors.Add(new AmazonSigningEndpointBehavior("ItemSearch", accessKeyId, secretKey));

There are three changes from our original console program:

1. The binding constructor now takes a BasicHttpSecurityMode.Transport argument. This forces an HTTPS connection. The Amazon web service refuses to accept authenticated requests over plain HTTP.
2. The endpoint address starts with “https:” instead of “http:”. The remainder of the URL did not change.
3. An instance of our endpoint behavior class is added to the client’s channel factory. This ensures that our message inspector will get plugged in when the client creates new connections, and will get to sign outgoing SOAP messages.

That’s it.

This is all that’s required to get your WCF based Amazon Product Advertising API application to send authenticated requests. If you build and run the application now, your console should list 10 book titles returned from the service.

Worked for you? Didn’t work? Let us know in the comments!

Read part 2.

Writing for OS/X or the iPhone? Using NSDictionary to store and access key-value pairs? Ever tried to enumerate them all only to find they come back in some random order?

That’s where I was several precious weeks ago. Of course you could get all the keys and all the values and then sort them all. But that takes CPU cycles, and is wasteful, especially if you have to re-sort after adding just a handful of items.

There are two popular classes of data structures used for implementing dictionaries. Hash tables are optimized for extremely fast gets, inserts and deletions, as long as you’re not changing the number of entries in the dictionary too dramatically. If you do, performance starts dropping, and the hash table has to be resized, the memory reallocated, and the content copied — a costly operation. As for sort order — there isn’t any. The order of elements in the hash table isn’t related to the elements themselves, and may change when the table is resized.

The other popular group of data structures are trees. In particular, self-balancing binary search trees specialize in reasonably fast gets, inserts and deletions. They don’t need to reallocate when some growth threshold is reached. Most importantly, they maintain a clear order between their elements. When used for building a dictionary, a binary search tree uses the sort order of the keys. You could quite efficiently enumerate all or part of the elements in the tree in their increasing or decreasing sort order.

Outside the Cocoa universe, both hash-based and tree based dictionaries are frequently available. Java has HashMap and TreeMap. The .NET Framework has the Dictioanry and SortedDictionary classes. C++ has the tree-based std::map and the hash-based boost::unordered_map. Cocoa’s NSDictionary and NSMutableDictionary are hash-based. If there is a sorted dictionary avaiable in Cocoa, it managed to hide quite well.

So after a prolonged bout of yak shaving, I am posting self-balancing binary search tree implementations of Objective-C SortedDictionary and MutableSortedDictionary classes. They implement the interface of NSDictionary and NSMutableDictioanry, and can be used as drop-in replacements. The new classes are unit-tested, and you can plug in their documentation directly into XCode. And, their sort order is always well defined. When you enumerate a SortedDictionary, you always get entries sorted by key. You can enumerate from the back too. If you query them for their keys and values, they come back sorted. And so on.

You can find the full source code, documentation and unit tests for SortedDictionary and MutableSortedDictionary at http://code.google.com/p/cocoa-sorted-dictionary/. They are available under the rather permissive MIT license, so feel free to hack away.

Oren Trutner