Collaborative Thinking

Atom + XBRL = Smart Move By The SEC

From Sam Ruby's blog:

The SEC started using RSS feeds two or three years ago to push information about XBRL filings received under the SEC's Voluntary Filing Program. See http://www.sec.gov/Archives/edgar/xbrlrss.xml

See the prior entry on this blog for a link to the iBlanket widget which makes use of one of the ATOM feeds.

You can go look at the Edgar system and see the ATOM feed here.  Look for the little orange icon, click on it.  You can read the feed in your browser, but the important thing to understand is that a computer application can read this also.

Basically, according to a IRWebReport blog entry, each filer to the SEC has its own ATOM feed which a user can subscribe to.  Imagine that the ATOM feed making you aware of information in an XBRL filing.  A computer application watches the ATOM feed, becomes aware of a new filing, reads the information from the XBRL filing, and updates your internal analysis models or other applications.  That is extremely powerful.

"The SEC, they totally get it." - Blog: Financial Reporting Using XBRL - XBRL as used for financial reporting by business users.

Atom Wins? Not Until The Fat Lady Sings

Some parts of Microsoft are adopting Atom/APP ... other groups remain somewhat vague and non-committal. It would be great for the SharePoint team to make a clear statement of direction on support for Atom and AtomPub. Given the growing deployment estimates of MOSS - which for some odd reason implemented only RSS - it seems that someone should commit to fixing an architectural "oops" .

But, put succinctly, Google + Microsoft = AtomPub wins. To paraphrase Dave Winer, the act of putting aside ego and saying a competitor's API is good enough, and that you're going to support it, is a brave and important act in the world of technology. That makes this convergence particularly exciting.

Anil Dash: Atom Wins: The Unified Cloud Database API

Building A Robust Feed Syndication Platform (Part 2)

Lawrence was kind enough to respond to my earlier post on the infrastructure NewsGator has put together to build out its feed syndication platform. I mentioned that I remain amazed at how poorly large vendors (e.g., IBM, Microsoft) are responding to this strategic middleware/infrastructure requirement that large enterprise customers will need as part of Web 2.0/Enterprise 2.0 efforts.

I stand by that statement.

Regarding Microsoft, I would refer people to this post where I summarize its efforts "Microsoft Announces FeedSync But Its Strategy Remains Un-Sync'd". It is true that Microsoft has implemented RSS support quite broadly throughout SharePoint. The important point business and IT decision makers is that the implementation is more application-like than infrastructure-like. Microsoft does little to nothing concerning the need to manage feeds from an end-to-end perspective (thus the partnership with NewsGator). Additionally, Microsoft needs to be much more clear on where it will be implementing Atom and AtomPub within SharePoint. The continued support only for RSS is architecturally a dead-end and reflects a poor upstream design decision.

Regarding IBM, they are almost as bad. There is no cohesive strategy coming from IBM concerning a feed syndication platform either. They do support RSS and Atom (in fact, IBM has been very aggressive with Atom/AtomPub and it is well implemented within Lotus Connections). But overall, its implementation of XML feeds across its product portfolio (e.g., Lotus Notes/Domino, Connections, Mashups) still does not meet the requirements of a feed syndication platform (as represented in the market by Attensa, KnowNow and NewsGator). There is some work being done that appears to be under-the-radar. At Lotusphere, IBM demonstrated a possible feed syndication platform in its Innovation Lab. There is also some interesting work being done by some IBM'ers around Abdera. Still, there is no clear product roadmap message from IBM regarding a feed syndication platform that could be deployed at the middleware layer. It also (like Microsoft), implements this capability as part of a product capability (which is more application-like than infrastructure-like).

Lawrence Liu
Senior Technical Product Manager for Community and Social Computing
Microsoft SharePoint Products and Technologies

We (the SharePoint team) understand feed syndication just fine, which is why we have RSS feeds for practically everything in SharePoint, and the feeds are easily configurable by the end user if necessary. SharePoint also has a built-in RSS Viewer web part to consume RSS feeds. As for feed aggregation, management, and sharing, we were hoping that the Exchange team would build that, but they had other priorities. NewsGator has been and will continue to be an excellent partner for us in this area.

Collaborative Thinking: Building A Robust Feed Syndication Platform

Building A Robust Feed Syndication Platform

Interesting case study of NewsGator's SaaS-based offering. While this type of scale may not be necessary within an enterprise per se - it does illustrate how such platforms should be approached as a core infrastructure decision. With only three enterprise-class vendors in this space (Attensa, KnowNow and NewsGator), I remain amazed at how poorly large vendors (e.g., IBM, Microsoft, Oracle) understand feed syndication and continue to treat it as an application decision. Stunning.

Solution Overview

http://www.newsgator.com

Customer Size: 65 employees

Organization Profile

Based in Denver, Colorado, NewsGator Technologies develops and markets solutions for the aggregation and viewing of Really Simple Syndication (RSS) feeds.

Business Situation

NewsGator needed to enhance the relational database infrastructure it uses to support 2.5 billion RSS articles totaling 4 terabytes, as part of its RSS aggregation and custom delivery solutions.

Solution

NewsGator is upgrading to Microsoft® SQL Server® 2008 Enterprise Edition (64-bit) database software running on the Windows Server® for 64-Bit Systems operating system.

Benefits

• High availability with Database Mirroring
• Reduced backup storage needs with Backup Compression
• Better control with Resource Governor
• Scalability
• Easier data management

Hardware

• Dell PowerEdge server computers with 4-way, 64-bit, dual-core processors and 32 GB of RAM

Software and Services
Microsoft SQL Server 2008
Windows Server 2008

Vertical Industries
IT Services

Country/Region
United States

Microsoft Case Studies: NewsGator

Info 2.0: Harnessing the power of Web 2.0 and Enterprise Mashups

Moderator:
Angelique Matheny, IBM Rational Software
Speakers:
Lauren Cooney, CTO Office, Information Management Group
Sriram Padmanabhan, Distinguished Engineer, IBM Advanced Tech. Group

Info 2.0 Webcast Notes (I assume that replay information will appear at the site):

The Good

• Need to move way from rigid applications to a more free-flowing approach is correct (synergistic with emergence, user-generated content, architecture of participation and other trends re: Enterprise 2.0)
• Also valuable to note that IT needs to create "safe sandboxes" for end users to create their own situational applications in ways that IT cannot predict or effectively support
• Excellent point on need for a transformation layer when talking about data, feeds, user / business context etc.
• Interesting idea on creation of specific data feeds (perhaps different than normal content-centric feeds) to help with mashup scenarios.

The Bad

• No connection of Info 2.0 message with other messages coming from Lotus. Odd given Lotusphere was just a few weeks ago.
• No product alignment or how this fits into the entire IBM portfolio (e.g., Quickr, Connections, etc).
• Slides point out Google Gadgets but IBM announced at Lotusphere that they were heading down the Eclipse path with iWidget.
• Mashup Hub is not a feed syndication platform (e.g., Attensa, NewsGator or KnowNow). The talking points made it sound like it was a real feed server and it's not really at that level of completeness, positioning what IBM is doing with managing feeds vs. other solutions in the market should have been made more clear.

And The Ugly

• IBM demo'd some feed server technology within its Innovation Lab at Lotusphere that could evolve into a feed syndication platform but it seems like this nugget of gold remains undiscovered at the management level.
• Which means IBM does not seem to strategically understand the importance and role of a feed syndication platform - given the need for enterprise organizations to have a cohesive middleware layer to manage feeds consistently - it remains astounding how major vendors continue to not have any clear direction or solution in this infrastructure area.

For Burton Group clients, we recently published a reference architecture template document on the architectural components of a feed syndication platform. A feed syndication platform:

• Functions as both a communication channel and hub
• Implements a publish-and-subscribe architecture that mediates consumers (e.g., applications that receive feeds) and providers of information delivered through feeds, often based on Extensible Markup Language (XML) (e.g., Really Simple Syndication [RSS] and Atom)
• Contains a collection of functional components that expose services and interfaces, which enable those components to interact with other applications, infrastructure, and environments external to the feed syndication platform
• Delivers feeds to destination points that exist on a variety of clients (e.g., PC, mobile), applications, and server platforms
• Integrates with applications that facilitate delivery and interaction with feeds, such as e-mail, web browser, instant messaging, custom applications, virtual workspaces, and portal sites
• Receives information from a variety of systems (e.g., enterprise content management systems, search engines, messaging platforms, and business applications) that it delivers as feeds
• Honors policy sources (e.g., permissions, rights management, and compliance) as they relate to information sources, feeds, and feed items, and applies such policies across the feed syndication platform (e.g., subscriptions, storage, download, and user management)
• Draws on related infrastructure platforms to complete the end-to-end system (e.g., directory, security, and integration services)

Jabber XCP Delivers Eventing via Pubsub

What will likely become a very critical component of "lifestreaming" that augments Atom/AtomPub (also a fundamental component):

Our developers are at it again, adding support to Jabber XCP for XEP-0163: Personal Eventing via Pubsub, which should be of interest to anyone following the evolution and use of presence technology. XEP-0163 lets users send updates about anything to users on their rosters. Personal eventing lets people easily publish things about themselves - it doesn’t get any more user-centric than this! The updates are sent using the XMPP Publish-Subscribe functionality used in Jabber XCP’s InfoBroker and described in XEP-0060. One way to look at it is that XEP-0163 takes XEP-0060 functionality to a more personal level.

...

The success of Twitter (which has XMPP in its architecture, BTW) and other similar services proves that personal eventing (in addition to presence, in general) is valued in social network settings. Service providers should be interested in the increased stickiness that personal eventing brings to their communities. Once users get used to seeing their friends’ moods, blog posts, activities, etc. they are more likely to stay in the communities which publish these details. The extension of social network technology to enterprise applications is in full swing, so by adding support for Personal Eventing via Pubsub to Jabber XCP, our extensible and highly scalable real-time presence and messaging platform will take the Power of Presence to a more personal and valuable level. The customers we’ve talked to about PEP have some great use cases and they will use this new functionality in their deployments. How about you?

Jabber Filaments Blog » Blog Archive » Jabber XCP Presence Platform Gets Personal (Eventing via Pubsub)

Google FeedServer

It will be interesting to see how this effort takes off and whether it has any impact on commercial offerings. (Discovered via James Snell's blog.)

Announcing Google FeedServer

Hello,
I'd like to announce a little project that we've been working on for a while
at Google and would like the feedback of the Abdera community. We're really excited about the potential of Abdera and have chosen it as the foundation for an open-source Atom Publishing Protocol Server. We hope that the approach we've taken is helpful to those who would like to create read/write Atom feeds for existing data sources.

There are more details below, but I just wanted to give a quick update on
where we'd like to go from here. We have lots of plans for the FeedServer, and would like to contribute back to the Abdera project. The FeedServer was conceived as a separate product and could possibly be an Abdera sub-project when it graduates from incubator. We are also considering how we will use it for the Data API portion of the Shindig project. For now, we have a project on Google's Codesite that should give people the opportunity to look it over and offer feedback:

http://code.google.com/p/google-feedserver/

Announcing Google FeedServer - Google FeedServer | Google Groups

An Open Source Atom Implementation

The goal of the Apache Abdera project is to build a functionally-complete, high-performance implementation of the IETF Atom Syndication Format (RFC 4287) and Atom Publishing Protocol (RFC# Pending) specifications.

Abdera is an effort undergoing incubation at the Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Apache Abdera

Social Presence: Time To Push The Reset Button

Just a collection of rough notes, thinking and musing on the next generation of "presence" that I've been wanting to assemble for a while:

• Presence has been historically been associated with a variety of communication technologies – IM, phone, mobile, etc. It is often associated with the consumer space where ICQ and AIM emerged as very popular tools that brought presence into the mainstream. You can go back further to find technical roots. Enterprise presence began in 1998 with IBM Sametime. Jabber emerged and made XMPP popular which has its own presence capability (2000-ish).
• People participate in presence systems because they wanted to broadcast their availability – to make other people, namely their friends, aware that they were online in case they wanted to contact them in some manner. You want to let people know that you were available – and contactable – through some type of communication tool.
• Presence started as a fairly simplistic concept. In the beginning, you usually had to manually set your presence – “away”, “busy”, “invisible”. Some tools would let you customize your status message. Tools became more intelligent and sometimes would derive your presence (lack of computer activity would change presence to “away”).
• "Aggregated presence” also emerged – other devices could be probed to influence your presence status. For instance, if you were on the phone, and the phone was connected to the presence system, you would be marked as “unavailable” or “on phone” until you hung up (which would cause your presence revert back).
• "Rich presence” was another advancement where presence was augmented with additional information about you. For instance, someone could click on your presence information and view a user profile (business card) that often displayed additional information (title, department, etc).
• Some presence systems were also smart enough to know what tools you were using or what part of a system you were in (“Mike is in the document library editing ABC file”).
• Presence continues to mature and now bumps into the Attention Management space as well. We want presence to help with attention management tools that deflect undesirable noise but let through relevant signals.
• People often talk about the killer app for presence which brings up presence-enabled applications and presence-enabled workflow (presence of a role entity vs. presence of a person). Presence indicators often appear inside an application context not just in the communication tool ("buddy list"). You are in an e-mail client and see that the sender is online – why reply when you can chat via IM or on the phone.
• Now we see more capabilities to categorize presence for instance, and reveal it to different groups (my team, my department, my company). And we can federate presence to outside entities (Yahoo!, AOL, Google, etc).
• IBM and Microsoft remain the key vendors for presence overall given their market strength but Jabber remains credible. Virtually all communication vendors remain anchored to SIP/SIMPLE. Few support XMPP. Jabber remains the champion for XMPP as the foundation for presence systems.
• Presence begins to become tied to location services now as well. Mashups are possible where you can show presence on a map. Mobile presence gets a boost from vendors like Iotum that show how an application can leverage presence.
• So we’ve been on this linear track of presence in terms of evolution and maturation – always anchored around communication – getting more intelligent – better aggregation of presence through multiple connections to other communication tools – richer and so on.
• Presence is a key underpinning for unified communications.

And now its time to get off this train track - time to push the reset button.

• Not completely, but the new perspective on presence is that it’s more social and should not be so strongly tied to communication tools for contact purposes. If you look at what’s going on in the consumer space and in social networking – the notion of a lifestream (like the Facebook news feed) and Twitter is a much more relevant model for think about a next generation “presence” framework in a broader sense. Presence is a stream not a static thing that flips once in a while when the system detects an event. We need to step away from presence as being fundamentally owned by UC and think of how it is used in a much more social context.
• "Social Presence" is the uber concept that will build around a feed syndication platform that needs to have backward support for existing presence solutions but opens up the door to many more non-communication activities to become part of what makes you “present”. Presence is really about what you are doing – part of that is for other people to know so they can connect with you but also just to know socially (e.g., ambient intimacy).
• That’s the point I’d like people to take away on presence – it’s beyond UC. It’s about the social aspects of how people observe and create a sense of being connected by having a means for people to let others know what they are doing, where they are, etc.
• Technically, it’s architecturally about a feed syndication platform of which communication tools can participate in but so can a large variety of other systems now locked out of presence systems. Traditional rich presence will continue because there are unique aspects of presence when it does come to UC. But the center of gravity will shift away from UC (and UC vendors).

Summary:

1. Presence is a key component of unified communications. Presence must seamlessly transition across mobile, IM, etc. Presence must be aggregated from the multiple devices and applications that we interact with. Presence must support richer information about us. Presence must be supported by more intelligent services that enable other solutions like attention management.

2. Presence is broken. It is owned by UC vendors that are not moving the technology forward. Presence makes too many assumptions (SIP/SIMPLE) and has a terrible data model. Vendors are not working with the standards and are not really interested in moving presence into something more open where many other systems and vendors can participate - given the importance of presence, vendors want to "own" the solution and leverage it for their own agenda.

3. People are moving onto the next generation of presence (Social Presence) and looking at social networking as the new model (Facebook). Presence needs to be more of a stream – no longer just about being online and available – more about what you are doing. This means pushing the reset button on presence from a UC and SIP/SIMPLE perspective. We need to think of presence more along the lines of a lifestream or activity stream where a variety of information is published into the stream and people can subscribe to the entire stream or to different types of information placed into the stream.

4. This next generation of presence will require a feed syndication platform that is likely built on top of Atom/AtomPub, microformats etc. Twitter and Facebook are the new models for this broader perspective on presence.

Microsoft Announces FeedSync But Its Strategy Remains Un-Sync'd

Personally, I find this very confusing. Microsoft has not articulated any coherent vision on feeds in general so my initial reaction is that this announcement strikes me as somewhat of a "one off". FeedSync (the evolution of Simple Sharing Extensions, or SSE), tries to solve more advanced problems that are outside mainstream adoption of XML feeds within enterprises right now. It also leaves Microsoft clients without a clear framework for how XML feeds and feed syndication comes together for those investing in the Microsoft platform. What I do believe is that we need to move away from feed synchronization being left up to individual vendors - so there is a clear need for a community-effort to standardize in this area (as mentioned by Sam Snell of IBM at the bottom of a blog post of the FeedSync topic). There is also the broader challenge of data synchronization (where tools like Groove and Notes have advanced replication engines that are unfortunately locked up inside those respective products).

Perhaps someone will take this spec and run with it - creating some interesting and innovative applications that can better showcase its value. But I wish Microsoft would fix some of the more basic gaps and glaring holes in how it is approaching XML feeds and feed syndication in general. Right now, "the cart is before the horse" so to speak.

Initially, Microsoft delivered the Windows RSS Platform as part of IE7. IE7 included its own lightweight feed reader (which I actually like, it does what it is supposed to do and no more). Windows RSS Platform (which I also like), was positioned as common client-side infrastructure to provide consistent feed-related services for desktop applications (e.g., feed subscriptions, download, storage).

Then the Outlook team undercut that effort by implementing (essentially), its own version of Windows RSS Platform as part of Outlook 2007. That's bad enough - but the implementation is absolutely horrible when you use it for a large number of feeds (BTW, my machine literally dies when Outlook 2007 syncs and despite deleting feeds several times, they keep coming back - very frustrating, sorry I turned the feature on to tell you the truth).

Then, you had essentially a failed effort to build a hosted feed platform with Niall Kennedy coming on board and then leaving.

SharePoint exposes a lot of information via RSS feeds but apparently has no support for Atom - in fact, Microsoft seems to be very unclear on its support for Atom and perhaps might prefer to play with RSS extensions that muddy the waters given RSS is essentially an architectural dead-end. SharePoint is not a feed syndication platform - it's just another application that exposes feeds. This gap forced Microsoft to partner with NewsGator (i.e., Social Sites), but even that integration does not eliminate the need for enterprise IT organizations to look at what Attensa, KnowNow and NewsGator offer themselves as complete feed syndication platforms.

Surprisingly, IBM is also completely absent regarding a feed syndication platform. I find it amazing (in an underwhelming manner), that a company touting social computing (e.g., Lotus Connections) and "Info 2.0" has not articulated a strategic vision related to XML feeds outside a simplistic client implementation in Notes 8 and surfacing XML feeds in its related back-end products (e.g., Domino, QuickR, etc). For now - Attensa, KnowNow and NewsGator remain the only credible options with perhaps Oracle as perhaps the only large vendor that could make a move here.

Synchronization for the Web

The creation of FeedSync was catalyzed by the observation that RSS and Atom feeds were exploding on the web, and that by harnessing their inherent simplicity we might enable the creation of a “decentralized data bus” among the world’s web sites. Just like RSS and Atom, FeedSync feeds can be synchronized to any device or platform.

Previously known as Simple Sharing Extensions, FeedSync was originally designed by Ray Ozzie in 2005 and has been developed by Microsoft with input from the Web community. The initial specification, FeedSync for Atom and RSS, describes how to synchronize data through Atom and RSS feeds.

...

Data synchronization is a key enabling feature for the software plus services world.

End users increasingly expect and want access to their data from servers, clients, and devices. They expect the data to always be up to date no matter where they access it, and users must never lose data in the process, or see duplicated items.

...

Everyone has data that they want to share: contact lists, calendar entries, blog postings, and so on. This data must be up-to-date, real-time, across any of the programs, services, or devices you choose to use and share with.

Too often today data is “locked up” in proprietary applications and services or on various devices. As an open extension to RSS and Atom, FeedSync enables you to “unlock” your data—making it easy to synchronize the data you choose to any other authorized FeedSync-enabled service, computer, or mobile device. FeedSync enables many compelling scenarios:

• Collaboration over the web using synchronized feeds
• Roaming data to multiple client devices
• Publishing reference data and updates in an open format that can be synchronized easily

...

What does FeedSync add to RSS and Atom?

RSS and Atom were designed as notification mechanisms, to alert clients that some new resource is available on a server. This is a great fit for simple applications like blogging.

But those feed formats are not a natural fit for representing collections of resources that change, such as a contact list, or a collection of calendar items. Atom Publishing Protocol is designed for resource collections, but it is a client-server protocol and isn’t suitable (by itself) for multi-master scenarios. FeedSync extends RSS and Atom so that FeedSync-enabled RSS and Atom feeds can be used for reliable, efficient content replication and multi-master data synchronization.

Windows Live Dev

Social Software Vendor Roundup

Quick partial listing of vendors that frequently come up in my client inquiries (either from clients themselves, or referenced by myself):

Category	Vendor/Product	Comment
Blogs
	Apache Roller	Open source, also used in IBM Lotus Connections
	BEA Pages
	Jive Software Clearspace	Blogs one component of platform
	Microsoft SharePoint Products & Technologies
	Six Apart, Moveable Type
	Traction Software TeamPage
	WordPress	Open source, backed by Automattic
Wikis
	Atlassian Confluence
	BEA Pages
	IBM	Wiki capability within Domino, QuickR and QEDwiki
	Jive Software Clearspace	Wiki one component of suite
	Media Wiki	Open source
	Microsoft SharePoint Products & Technologies
	Mindtouch Deki Wiki	Open source community at OpenGarden.org.
	Socialtext
	Traction Software TeamPage
	Twiki	Open Source
Social Bookmark Systems
	BEA Pathways
	Cogenz
	Connectbeam
	IBM Lotus Connections	dogear component
	Scuttle	Open source
Feed Syndication Platforms
	Attensa
	KnowNow
	NewsGator
Social Network & Community Sites		Typically offer a mix of user profiles, blogs, wikis, social networking, etc.
	Awareness Networks
	CollectiveX
	Communispace
	HiveLive
	IBM Lotus Connections
	iCohere
	KickApps
	Lithium
	Microsoft SharePoint Products & Technologies
	Ning
	Ramius CommunityZero
	Select Minds
	Sparta Social Networks
	Prospero
	Telligent Community Server
	Tomoye
	Wetpaint