Aaron’s Potlatch

So, I saw the article about Cisco entering the server, or private cloud market.  The big enterprise vendors Microsoft with Azure, Sun with Q-layer, have started to want to get in on the cloud hosting companies and several announcements recently have been interesting.  However, in my opinion the market leader is still Amazon Web Services, EC2, S3, etc.  I recently moved my Demisauce server to EC2 and really enjoyed setting it up.

As more and more capability moves to hosted services, open-source gains with linux etc but what else?   Xen was gaining especially with Amazon EC2, however Ubuntu seems to be moving away from xen towards KVM, as does Redhat  (Wow, kvm just needs to get faster though, I run it on my quad core AMD machine at home and it is much, much slower than my VMWare fusion images on my mac).  Redhat is building everything it needs for a CloudOS, with virtual machine management (Ovirt), hypervisor (KVM), security & directory (FreeIPA), and an API for virtual machine configuration (libvirt).

Once again, I can’t help but feel that the value proposition of Open-source is being missed by some of the big-co’s.  Open-source is so valuable because capability and inventions by one can be utilized by other’s.  This really isn’t as possible (ok, you can license technology from each, but that doesn’t happen much) among the proprietary vendors.  So, in the end run I think that the open source stack is going to exceed what the vendor’s provide.

I just posted an article about the new Comment system for the blog i am working on.  Full article here…

Here is a small picture, the post has full set of pictures.

Recently I have been working on Demisauce, an open source Software as a Service project to provide services to other applications. Some colleagues of mine were trying to solve a problem at work that involved integration of voting and commenting capability into an existing software application (.NET) using a service approach.  Meanwhile, i was working on a tool to allow a “help overlay” to overlay parts of a web page with a dialog type box of content managed by Demisauce but shown within the browser of a page served by my application.

While talking about where Demisauce is headed, the need to incorporate “services” that are both server to server communication, as well as browser to server became apparent.

Diagram showing services that are both browser to server, and server to server.

With the browser more and more being the controller of MVC architecture, it is aggregating content (mashup) into  a single view and application.  However, that content comes from multiple servers.   However, again, the need to have services such as commenting applications, comments, help, etc that are usable within your web application as a “private” application (internal corporate software applications) instead of the web2.0 consumer/public model prevalent in mashups today.

The services we have targeted initially for this are the help application, to be able to pop open an overlay/dialog box of “help” contextual to your page or application, but have the content for that managed on Demisauce.  To allow for feedback on pages, and the management of that feedback (has anyone processed it), for generic content, email templates, etc.

In creating software, we often are faced with build vs buy vs combination (integrate). While creating a software application, I find that we have a series of patterns that repeat itself related to needing features that are “non core”. Some examples:

• One of our teams an application needs a “Tiny url type functionality” only internal, secure, private to have less brittle url’s. We have ~30 different web based applications that make up our environment, and invariably we break others when we update one and the “link api” changes. We want a means of indirection, so if the end “link api” changes, we can change it on the central “link server” instead of updating the other 29 software applications. See info on PURL’s and a recent project to update the actual PURL sever.
• We also need content (news) to show beside our custom software on the right side context. We need to notify the users of changes to the software, impending outages/upgrades, etc. Since these don’t happen that much we update them statically since we are so focused on “Core” functionality.
• We send out email, and find ourselves creating email templates stored in our own db, or email template management admin tools also.
• We also find ourselves creating a Help system to author little tips/help built into our application. None of these do i consider “Core” to why our application exists. The business

Diagram showing the Core vs. Non core aspects of our applications.

In Addition, we have received numerous requests for other features that have just been to low on the priority queue to ever find time to add to our systems also shown in above under non core aspects of applications.

So, Demisauce’s charter is to provide very, very easy to use through library integration with a server that has admin tools to edit content, but that content and services are utilized by applications via web services.

Over the last few years, our work experience as software engineers has changed pretty drastically, or at least where I work. More and more web services and application integration is taking place. Third party apps offer web service connectors, there are large numbers of technologies for doing integration from ESB/BPM/BPEL to ETL or Change Data Capture. As more and more corporate development and software development goes to Software As a Service (Salesforce.com, Amazon S3, Google Apps), Open Source in my opinion faces a challenge of remaining relevant. WordPress.com doing a combination model of open source software and then a hosted version of it is an example of the struggle ahead.The traditional choices of software look like this (gross approximation but it works).

Diagram 1: Software development stack options of in house development with Open Source libraries, or Open Source framework with customizations.

You seemed to have a choice in doing software development to choose a series of libraries or a framework Libraries: Hibernate, Jboss, Struts in the Java world, Ruby on Rails, Typo etc in the Ruby world, or, Django, Zope, Plone in python Frameworks: Joomla and a large number of PHP apps, or even Sharepoint in the non OSS MS world. However, I think if other people’s experience are like mine, trying to get a “CMS” framework to fit into your needs was difficult. Most Frameworks, Portals, and CMS sytems fall into a trap of trying to be a better mouse trap and asking “if only you use my system, it will be good”. CMS, Portal, or App frameworks talk about “Plug-in” architecture, but still this forces you to a generally cumbersome development paradigm compared to building your app from library components.

At my workplace, our applications were custom software development applications, but it would have been nice to have the built in general navigation/menuing, page management, content management of the CMS frameworks to show messages within the application. However, to build on top of the CMS framework was much more cumbersome for doing software development.

So, DemiSauce is software that is only “half of a software stack” (Hence the name: Demi for half, Sauce for Software As a Service SaaS). It is not meant to be a stand alone web application. It is meant to be able to provide web services to your custom software applications giving them ability to get and display “content” such as “news”. It is meant or assumed to also have a cache system between the application, and DemiSauce servers as well. This allows content and services to be shared across multiple custom software applications.

The Services that are currently being kicked around are:

• Content Management:
  • GetListOfContent for a context (page/area of site)
  • Knowledge base, or help system authoring
• Feedback tools: “Submit feedback here…”
  • “Rate this page or app”

DemiSauce is a Pylons Web Application built with SqlAlchemy and mako templates.

My work environment is a very large distributed set of web applications. There is a large number of applications, and it seems that we continue to face the issue of rebuilding the wheel for common services.  Many OSS web applications (CMS systems etc) may be suitable, but they all seem to fall victim to the “I am the center of the Universe” design philosophy. This is, “if you use me to build all your stuff, all will be fine”. However, this never happens. We have 3rd party wiki tools, 3rd party CRM. We have custom home grown legacy apps, Microsoft sharepoint. One group took Radiant CMS and installed it, then get a data feed from it to display “news” about their application.

Also, we are building a Menuing application at work that has hierarchical capability. Since many different apps (Some java, some .NET, some 3rd party) are all trying to appear to the end user as a single experience it would be nice to be able to update the shared menu items in one place and have the changes propagate.

The solutions today seems to be “use our app”: Sharepoint has menuing and in theory could offer a web service but that seems like overkill. Plone, the Portals, etc all seem to be focused on getting the applications built on top of their infrastructure.

So, I think the reverse, a set of services which are not intended to be “built on top of” but rather accessed through libraries. Think Amazon S3 services.

• CMS System: Admin to log in and edit, add articles and content (About us, policies, etc). Then the display would be shown within your web application by calling a library (over WS). This assumes you would have caching on your end.
• KB system. Knowledge base/help, again templated for your own system. It looks like your site, and is your site. But, having an admin to go login to and change the content, articles can be shared across sites. I know some of the commercial KB systems have web service API’s, so this would be similar.
• Menuing system. Ability to update navigation and it will update on multiple seperate applications.