The Keyword Services Platform (KSP) is a keyword research tool available through Microsoft adCenter, which contains a set of algorithms for providing information about keywords used in search engine queries.
The KSP was originally conceived by ZhaoHui Tang, Dylan Huang, Wayne Guan, Jiong Feng, Li Luo, Ken Kwok, Fred Nie at Microsoft adCenter Labs in May 2006. It underwent a major overhaul in 2011 and the platform as we see today was developed by Nimeesh Patel, Shravana Aadith Ramia Bapulal and Vivek Vinodchandra Pradhan. The platform aims to provide a core set of data and technology to empower search engine marketing and keyword research efforts. The KSP uniquely delivers a standardized set of keyword technologies through a Web services model, accessible via an application programming interface (API) and a Microsoft Excel add-in.
KSP API beta access is available for researchers and developers upon request from the Keyword Services Platform feedback link.
Architecture The following components comprise the Keyword Service Platform architecture: Keyword API. Set of standard Web services for various keyword tasks. These services are based on the Windows Communications Foundation and can be consumed by client applications (e.g., Microsoft Excel add-in) or mashups. Provider Plug-in Framework. System that allows researchers to incorporate new algorithms or data mining models within the Service Container. Each provider brings a specific keyword technology to the KSP architecture—for instance, keyword association, keyword extraction, or keyword classification. Security. The component that handles permissions that pertain to provider procedures (methods) and Stored Procedure implementation. Server Object Model. The data model allows developers to leverage different services. Service Container. Set of providers that support various keyword technologies. Host to all service providers and Stored Procedures, which allows parallel execution. Shared Services. Core components, consisting of a crawler, in-memory data structures, word stemming algorithms, etc. These services are used by different providers and executed by stored procedures. Stored Procedures. Procedures for consolidating and centralizing the logic behind applications. Selected sets of these procedures are made available to users.
Developers may use .NET programming languages to create procedures that combine the use of different providers or implement additional business logic processing based on the output from a provider.
Keyword API The Keyword Service Platform has defined a set of APIs for each class of keyword services. These interfaces for Web services include keyword extraction (ITermExtraction), keyword categorization (ITermCategorization), keyword suggestion (ITermSuggestion), keyword forecast (ITermForecast), keyword monetization (ITermMonetization), and several others. The APIs define the signatures of each Web service.
Keyword suggestion Keyword suggestions are handled via the ITermSuggestion interface. To find the five most closely related keywords to "BMW", the following method call may be used: GetTermSuggestion("BMW",5). The query result is shown in the following table, and by default, sorted by confidence:
OriginalTerm Term BMW Auto BMW Car BMW Lexus BMW BMW cars BMW BMW Z4
To view the five suggested five terms with the corresponding confidence score, a third parameter can be used to indicate that statistics should be returned: GetTermSuggestion("BMW",5,true). The query result is shown in the following table along with columns for score and support. The results are similar to those available through the Data Mining Extensions (DMX) in SQL. Score represents the confidence or probability; support represents the number of cases supporting the rule in the training dataset.
OriginalTerm Term Score Support BMW Auto 0.96 10000 BMW Car 0.89 9000 BMW Lexus 0.89 11000 BMW BMW cars 0.83 12000 BMW BMW Z4 0.78 12800
To return only those terms with a high confidence score, a filter can be used on the Score column with the following method call: GetTermSuggestion("BMW",5,true,"Score>0.8"). The query result is shown in the following table. In this case, only four rows are returned, as these are the only terms that meet the criterion of the filter.
OriginalTerm Term Score Support BMW Auto 0.96 10000 BMW Car 0.89 9000 BMW Lexus 0.89 11000 BMW BMW cars 0.83 12000
When the table of terms possibly includes thousands of keywords, batch query syntax can be used. For example, suppose that the keywords are stored in myInputTermTable, and only the two most relevant terms for each keyword should be returned: GetTermSuggestion(myInputTermTable,2). The query result is shown in the following table.
OriginalTerm Term BMW Auto BMW Car Honda Lexus Honda Sedan Ford Pickup Ford Truck
Keyword demographics Keyword demographics are handled via the ITermDemographics interface. To obtain the demographic distribution for the keyword "Minivan", the following method call could be used: GetTermDemographics("minivan"). The query result is shown in the following table.
Term Male Female 0–13 13–18 18–25 25–35 35–50 50–65 65+ Minivan 0.40 0.60 0 0 0.1 0.2 0.4 0.2 0.1
Keyword monetization Keyword monetization values specific to paid search are handled via the ITermMonetization interface. The following method call returns the KPIs for the keyword "Online bank" based on the previous week's paid search data, in the third position of sponsored listings: GetTermKPIs("online bank",TimeInterval.LastWeek,3). The result of the query is shown below, containing the input keyword, the number of clicks in the sponsored link for "Online bank", overall impressions for the keyword, position, average click-through rate (CTR), and average cost per click (CPC).
Term Clicks Impressions Position CTR CPC Online bank 42 2915 3 0.014 1.325
Keyword extraction Keyword extraction is handled via the ITermExtraction interface. The following method call extracts the eight most relevant keywords from the webpage "autos.msn.com", and provides the corresponding statistics: GetTermExtraction("autos.msn.com",8,true). The result of the query is shown below, where the Score column represents the relevance of the extracted keyword to the page content, while the Support column represents the number of occurrences of a keyword on the page.
URL Term Score Support autos.msn.com auto reviews 0.62 3 autos.msn.com MSN autos 0.54 2 autos.msn.com cars 0.48 5 autos.msn.com sport cars 0.39 2 autos.msn.com used cars 0.38 1 autos.msn.com compare car 0.34 1 autos.msn.com new cars 0.32 1 autos.msn.com luxury cars 0.30 1
Sample code The following code fragment connects to the Keyword Services Platform server and uses the keyword term forecast Web service.
using (KeywordServer server = new KeywordServer("https://ksp.microsoft.com")) { server.UserName = "username"; server.Password = "********"; ITermForecast provider = null;
try { server.Open(); // Context can be set if needed. It will remain during the following calls. provider = server.GetProviderByImplementation<ITermForecast>("Microsoft.adCenterLabs.Providers.KeywordForecastProvider"); if (provider != null) { // Single mode API DataTable result = provider.GetTermForecast(term, -5, 3); DisplayResults(result);
// Batch mode API result = provider.GetTermForecast(terms, -5, 3); DisplayResults(result);
catch (FaultException) { // Handle fault returned from calling the proxy method
catch (CommunicationException) { // Handle lost network connection error
catch (TimeoutException) { // Handle time-out error
finally { if (provider != null) server.ReleaseService(provider);
Providers Each Keyword Services Platform provider supplies a specific type of keyword technology by implementing one class of a specific keyword interface (e.g., ITermSuggestion, ITermForecast, ITermExtraction). The API defines the signature of each Web service and the format of the returned data. The KSP provider is a server-side object encapsulating a particular implementation of a keyword technology. This provider exposes its functionality through service contracts in the Windows Communication Foundation (WCF). The WCF is Microsoft's unified programming model for building service-oriented applications, which enables developers to build secure, reliable, transacted solutions that integrate across platforms and interoperate with existing investments. To enable seamless integration of a provider into the KSP, and correspondingly seamless integration with third-party tools and applications, the providers must meet several conditions: custom configuration settings stored in configuration files, rather than the source code; standard .NET tracing and message logging to enable service monitoring and diagnostics; standard Windows Management Interface performance counters for performance monitoring; and document service contracts included using a service description language for better understanding and testing of the Keyword Service Provider.
Stored procedures Developers can write stored procedures (sprocs) using any .NET programming language. These procedures are executed on the Keyword Services Platform server, which hosts the Common Language Runtime (CLR). Similar to a database sprocs, a KSP sproc is designed to enable developers to implement several types of business logic on the server side after retrieving result data from providers. KSP sprocs do not require configuration management or setup requirements.
Two types of stored procedures are supported: Managed Assembly Stored Procedure (MASP) and Common Language Runtime Stored Procedure (CLRSP). A MASP consists of a compiled .NET assembly containing a public interface exposed through the KSP as well as any dependent files. Once the MASP is uploaded to the KSP through its management interface, it becomes callable by KSP client programs. A CLRSP consists of a source file written in one of the supported CLR programming languages (C#, Visual Basic .NET, Managed Extensions for C++, and others). The functionalities of the CLRSPs are exposed through a public interface defined in the source file. Once the CLRSP is deployed to KSP through its management interface, it is compiled on-demand by KSP and becomes callable by KSP client programs. Compared to database sprocs, KSP sprocs are object-oriented. A sproc may contain a set of related functions, or even identically named functions with different signatures.
Server Object Model and Shared Services Keyword Services Platform Server Object Models and Shared Services enable KSP Service Providers and stored procedure developers to access server-side objects and functionalities easily and consistently. The object model consists of the following three collections:
Service providers: This collection enables callers to access server-side Service Provider objects by name, implementation interface, and/or class name. Once callers obtain the Service Provider object, all of the functionalities of the service provider are accessible through its public interface. Stored procedures: This collection enables callers to access server-side Stored Procedure objects by name, implementation interface, and/or class name. Once callers obtain the Stored Procedure object, all of the functionalities of the stored procedure are accessible through its public interface. Services: This collection enables callers to access server-side shared services by name, by implementation interface, and/or class name. Once callers obtain the shared service object, all of the functionalities of the shared service provider are accessible through its public interface.
Cloud server model