dave's wiki

Configurable Pings for Multiple Sites#

Roller currently supports sending weblog pings only to weblogs.com. There is a frequently re-iterated standing request for enhancing this support to allow for pinging multiple sites and for making the site URLs configurable. This proposal page is meant to facilitate converging on a design for that functionality.

See also: ROL-305

AnilGangolli is facilitating this proposal.

Assumed Functionality#

At least the following functionality is expected:

• allow configuration of ping URLs either centrally or per-website or both
• allow per-website configuration specifying which URLs are to be pinged automatically whenever a new/updated entry is published
• allow per-category configuration specifying which URLs are to be pinged automatically whenever a new/updated entry is published in that category. A given URL will be pinged automatically if either the website-wide or per-category configuration indicates it should be.

Requirement Questions#

There are a few requirement questions that need to be settled in order to proceed with a design.

Central or Per-User Configuration of Ping URLs?#

The basic question is who can add a new ping URL? Should the set of possible ping URLs be configured centrally by an administrator? Individually and independently by each user (website)? or both (with each user inheriting a base set configured by the administrator)?

I'd like to keep this simple, which is my only reason for not immediately designing with the third option.

Protocol Flexibility?#

Is it sufficient to support only the weblogUpdates.ping protocol? What provisions do we need to make now for additional protocols?

For example, blo.gs defines a weblogUpdate.extendedPing method, but it is possible to avoid its use entirely by using their interactive form once and subsequently using the "standard" ping mechanism. Should we make any provision to support this or future variants?

While it is easy to provide pluggable protocol flexibility, each would potentially require different configuration UIs.

Optional arguments in weblogUpdates.ping#

Even the weblogUpdates.ping mechanism has two optional arguments (changesUrl and category) that are commonly unused. Do we need to allow for specifying those?

Per-category Automated Pings?#

Some aggregators, like java.blogs are topic-specific and people (like AnilGangolli, anyway), set up category-specific feed URLs with these services to avoid providing off-topic posts. It would be most appropriate to just ping java.blogs on updates to posts in that category. Things will still work if I "over-ping," that is, if I ping java.blogs on 'every' post; the only harm done will be unnecessary ping traffic, addition conditional GETs from the aggregator, and somewhat more load for the processing of these on both sides.

The question here is really about whether this extra impact of these unnecessary pings is significant enough to warrant restricting the pings to specific categories. Note that the impact is aggregated over all parties behaving this way, but it only happens on blog posts/updates.

Asynchronous Ping?#

The current limited ping fucntionality is synchronous with update/posting. The thread that processes the user request to save the updated weblog entry also performs the ping but currently there is at most one ping to perform. As we allow for additional pings, do we want these to remain synchronous with the post, or should we queue them for asynchronous processing?

I'm inclined to queue them and to do so persistently. See the next section as well.

Reliability?#

There are questions about ping reliability. Here are my basic assumptions. I think we should support a persistent ping queue so that update notifications are not lost on server restarts. We should also support re-queue on connection-related ping failures a (configurable) limited number of times.

Changes.xml?#

I may be misunderstanding the functionality involved in supporting a "changes.xml" query, but I think it is independent of the ping functionality. Roughly speaking, I think it could be supported today using a query operation based on weblog entry update times. I'm treating that as a separate topic, and outside the scope of this particular proposal.

Does Ping-o-Matic make this irrelevant?#

Given the advent of Ping-o-Matic do we still need this functionality? I've assumed that we do not want to rely on the presence of such services, so we still need this functionality.

Proposed Design#

Assumptions#

• We will initially support only the "de facto standard" weblogUpdates.ping XML-RPC over HTTP mechanism. Additional mechanisms will require addition of a separate field indicating the mechanism and probably additional configurable attributes that are specific to the mechanism. This might be reasonable to do as an xml-serialized field of attributes, or one might wish to do it as a separate table of attributes. The current mechanism will only support weblogUpdates.ping and configuration of the ping URL.

Model#

• Introduce the following new tables and corresponding data objects:

• • ping target: Each row in this table will correspond to a ping target. Ping targets can be common targets (defined centrally by an administrator) or custom (defined by a user). A custom ping target has an associated website. Initially this table would be populated from the XML configuration data with well-known ping URLs for the popular aggregation sites.

create table pingtarget {
    id           varchar(48) not null primary key
    name         varchar(255) not null -- short descriptive name of the ping target
    pingurl      varchar(255) not null  -- URL to receive the ping
    websiteid    varchar(48) null -- if not null, this is a custom target defined by the associated website
}

• • ping configuration: Each row in this table will correspond to an automated ping configuration, that is the indication that the server should ping a specified target on updates to a specified weblog.

create table pingconfig {
    id            varchar(48) not null primary key
    websiteid     varchar(48) not null -- fk reference to website for which this configuration applies
    pingtargetid  varchar(48) null -- fk reference to a ping target or null if a custom url is supplied
}

• • ping category relation: Each row in this table will correspond to a category for which the ping should be performed. If there are no association records the ping is interpreted to be performed for posts to any category.

create table pingcategory {
    id            varchar(48) not null primary key
    pingconfigid  varchar(48) not null -- fk reference to ping configuration
    categoryid    varchar(48) not null -- fk reference to category
}

• • ping request queue: Each row in this table will correspond to a queued request for the server to perform a ping. The entries are originally queued at the time the weblog entry is saved. A separate thread processes the queue.

create table pingqueue {
    id          varchar(48) not null primary key -- strict sequence number
    entrytime   timestamp not null -- timestamp of original entry onto the ping queue
    ping        varchar(48) not null -- fk reference to a ping config
    failures    integer not null -- number of failed ping attempts that have occurred
}

UI#

Here are the changes anticipated for the UI.

Configuration of well-known ping targets: A new page Settings | Admin | Ping Targets will be added for administrators to configure well-known ping targets. The page will list existing ping targets and will allow editing or deleting them individually, similar to the bookmark page. An Add a new ping target link will lead to another page with a form to allow editing the name and URL of the ping target and saving. The same form would be used for editing an existing ping target.

Configuration of pings, custom targets, and category restrictions: Introduce a new menu option Settings | Weblog | Pings and corresponding configuration page for users to set up their ping configurations. The new page will list current ping configurations, and allow the user to edit or delete them individually. An Add a new ping configuration link will lead to another page for editing or adding a new ping configuration.

The add/edit page for a ping configuration will display the well-known ping targets with checkboxes and will also display a checkbox specifying "Custom Ping Target" and with a text field that allows specifying the URL. Below that will be the user's category list with corresponding checkboxes. The user specifies a target URL (either well-known or custom) and zero or more categories (using the checkboxes); the instructions will indicate that if no categories are specified then the ping will occur on posts in every category.

So there will be two sets of ping targets, 'admin' and 'per user'? -LanceLavandowska
AnilGangolli: From the UI perspective, yes. Admin (or well-known) ping targets configured by the administrator, and Custom (per website) ping targets configured by a regular user. The present design proposes that they be stored in one table/model class (the pingtarget table) with custom targets distinguished by having a non-null websiteid. A regular user will be able to choose from among the admin/well-known ping targets as well as adding custom targets of his/her own. This will become clearer once I have a prototype working.