Outsource your non core portions of your application

In creating software, we often are faced with build vs buy vs combination (integrate). While creating a software application, I find that we have a series of patterns that repeat itself related to needing features that are “non core”. Some examples:

• One of our teams an application needs a “Tiny url type functionality” only internal, secure, private to have less brittle url’s. We have ~30 different web based applications that make up our environment, and invariably we break others when we update one and the “link api” changes. We want a means of indirection, so if the end “link api” changes, we can change it on the central “link server” instead of updating the other 29 software applications. See info on PURL’s and a recent project to update the actual PURL sever.
• We also need content (news) to show beside our custom software on the right side context. We need to notify the users of changes to the software, impending outages/upgrades, etc. Since these don’t happen that much we update them statically since we are so focused on “Core” functionality.
• We send out email, and find ourselves creating email templates stored in our own db, or email template management admin tools also.
• We also find ourselves creating a Help system to author little tips/help built into our application. None of these do i consider “Core” to why our application exists. The business

Diagram showing the Core vs. Non core aspects of our applications.

In Addition, we have received numerous requests for other features that have just been to low on the priority queue to ever find time to add to our systems also shown in above under non core aspects of applications.

So, Demisauce’s charter is to provide very, very easy to use through library integration with a server that has admin tools to edit content, but that content and services are utilized by applications via web services.

Service Oriented Software and OSS

Over the last few years, our work experience as software engineers has changed pretty drastically, or at least where I work. More and more web services and application integration is taking place. Third party apps offer web service connectors, there are large numbers of technologies for doing integration from ESB/BPM/BPEL to ETL or Change Data Capture. As more and more corporate development and software development goes to Software As a Service (Salesforce.com, Amazon S3, Google Apps), Open Source in my opinion faces a challenge of remaining relevant. WordPress.com doing a combination model of open source software and then a hosted version of it is an example of the struggle ahead.The traditional choices of software look like this (gross approximation but it works).

Diagram 1: Software development stack options of in house development with Open Source libraries, or Open Source framework with customizations.

You seemed to have a choice in doing software development to choose a series of libraries or a framework Libraries: Hibernate, Jboss, Struts in the Java world, Ruby on Rails, Typo etc in the Ruby world, or, Django, Zope, Plone in python Frameworks: Joomla and a large number of PHP apps, or even Sharepoint in the non OSS MS world. However, I think if other people’s experience are like mine, trying to get a “CMS” framework to fit into your needs was difficult. Most Frameworks, Portals, and CMS sytems fall into a trap of trying to be a better mouse trap and asking “if only you use my system, it will be good”. CMS, Portal, or App frameworks talk about “Plug-in” architecture, but still this forces you to a generally cumbersome development paradigm compared to building your app from library components.

At my workplace, our applications were custom software development applications, but it would have been nice to have the built in general navigation/menuing, page management, content management of the CMS frameworks to show messages within the application. However, to build on top of the CMS framework was much more cumbersome for doing software development.

So, DemiSauce is software that is only “half of a software stack” (Hence the name: Demi for half, Sauce for Software As a Service SaaS). It is not meant to be a stand alone web application. It is meant to be able to provide web services to your custom software applications giving them ability to get and display “content” such as “news”. It is meant or assumed to also have a cache system between the application, and DemiSauce servers as well. This allows content and services to be shared across multiple custom software applications.

 

The Services that are currently being kicked around are:

• Content Management:
  • GetListOfContent for a context (page/area of site)
  • Knowledge base, or help system authoring
• Feedback tools: “Submit feedback here…”
  • “Rate this page or app”

DemiSauce is a Pylons Web Application built with SqlAlchemy and mako templates.

Open Source Services

My work environment is a very large distributed set of web applications. There is a large number of applications, and it seems that we continue to face the issue of rebuilding the wheel for common services.  Many OSS web applications (CMS systems etc) may be suitable, but they all seem to fall victim to the “I am the center of the Universe” design philosophy. This is, “if you use me to build all your stuff, all will be fine”. However, this never happens. We have 3rd party wiki tools, 3rd party CRM. We have custom home grown legacy apps, Microsoft sharepoint. One group took Radiant CMS and installed it, then get a data feed from it to display “news” about their application.

Also, we are building a Menuing application at work that has hierarchical capability. Since many different apps (Some java, some .NET, some 3rd party) are all trying to appear to the end user as a single experience it would be nice to be able to update the shared menu items in one place and have the changes propagate.

The solutions today seems to be “use our app”: Sharepoint has menuing and in theory could offer a web service but that seems like overkill. Plone, the Portals, etc all seem to be focused on getting the applications built on top of their infrastructure.

So, I think the reverse, a set of services which are not intended to be “built on top of” but rather accessed through libraries. Think Amazon S3 services.

• CMS System: Admin to log in and edit, add articles and content (About us, policies, etc). Then the display would be shown within your web application by calling a library (over WS). This assumes you would have caching on your end.
• KB system. Knowledge base/help, again templated for your own system. It looks like your site, and is your site. But, having an admin to go login to and change the content, articles can be shared across sites. I know some of the commercial KB systems have web service API’s, so this would be similar.
• Menuing system. Ability to update navigation and it will update on multiple seperate applications.

New Collaboration Application

I started working on my new application recently, and just what the world needs, another collaboration application. However, this is targeted for Management Systems. I have been spending a lot of time thinking about this since my early career at Intel which had strong management systems.

Technology is going to be Pylons (with Mako templating and SQLAlchemy). Javascript right now is Yahoo YUI but, it is pretty verbose and heavy. Considering JQuery. I originally considered Ext.JS but that was very heavy(463kb for all) and too focused on widgets, not enough on the library. Also looked at Mootools, Prototype, MochiKit, Dojo, etc. Prototype and Mootools were too rubyish, prototype from my days with Rails didn’t seem as adaptable as Yahoo YUI. Since this is an “application” my thought was the JS would be cached. We will see if that holds up.

Collaboration context, or what is “IT”?

There has been much discussion about email as a collaboration tool, and possible alternatives. These are great discussions and really got me to thinking as well. It seems email started out as a communication tool, and as other information systems become more and more common it was used more as a tool to fill in the gaps of these information systems (“hey, please review this document and get back to me”, “can you login and approve my expense report”, “did you see the stats on the so and so report?”, “the xyz system just generated a system alert, more information here…”). An over generalization is that email discussing something that has no electronic information (as was the case with most things 20 years ago) works good (not great). Email as a bridge between the gaps in other systems doesn’t work as well.

Michael Sampson discusses the importance of correct usage and people issues, and it seems to me a large influence on this is what is “IT” that you are talking about? I think much of the email in my inbox is about some other system (see diagram below “Application Context”). In this situation, the person is switching context in order to process the email, they must also go to another system or have access to additional information. This diagram holds true for people sending notification of wiki pages to people on email as well. However, using Mashups, it seems possible to take the existing tool of email and allow it to be shown within the context of another application (see diagram 2). This would require some ability to tag, or differentiate emails for a specific context (for instance, a purchase order going through the request chain could have all the emails about this shown on the same page).

I tried out the new Gliffy for these diagrams, it worked great, thank you!

Thanks also to these articles for helping stimulate my thoughts here:

http://www.innovationcreators.com/2006/09/email_is_critical_to_enterpris.html
http://www.annezelenka.com/2006/09/email-the-good-enough-collaboration-tool
http://www.grey-consulting.com/blog/2006/09/email_the_good_enough_collabor.html
http://www.michaelsampson.net/2006/12/email_vs_collab.html
http://www.mindthis.net/mindthis/2006/12/counterpoint_em.html
http://confusedofcalcutta.com/2006/09/10/on-social-software-and-consensus/
http://www.inc.com/magazine/20060901/column-freedman.html