Thought Process

the product of systematic series of mental activity.

Monday, November 09, 2009

When coffee tastes bitter, it’s time for change

Few years back, I was enjoying a cup of hot coffee with my friends in office pantry during late evening break. One friend said, "This Coffee tastes bitter", I replied "When coffee from office vending machine tastes bitter, it’s time for change". Topic ended after some discussion. Last week I received a mail from the same friend "Coffee tastes bitter, as you said I should start looking out". I was surprised that he remembered my comment for so long.

Though I didn’t give so much thought to what I said then but when I pondered again it made sense instantly, I could immediately think of some metaphoric indicators for need of change from bitter coffee.
The change may not be necessarily employer change, it can be project or team or any other change suitable to individual circumstances.

So why coffee starts tasting bitter, first simple reason, you are drinking too much lately, which might link to too many breaks during office hours or too stretched work hours. Both the scenarios require some kind of change to keep things in order in long run professionally as well as personally.

Another reason I can think of is, vending machine coffee was always bitter but you started realizing it now, probably because previously you were content with your work and team environment, your efforts were well acknowledged and appreciated, many more things like that so you never really cared for coffee taste but enjoyed every bit of time spent at work but now things are getting little different.

Well managed and executed project promotes team taking break together sharing ideas and cracking jokes to keep motivation high and ball rolling. It also serves as platform for developing non formal relationship with co workers thus time spent in break mattered while coffee just served as ice breaker.
On the other hand, fractured teams also take coffee breaks probably even more frequent than above ones but mostly in groups of hierarchy , venting out concerns and issues in wrong forums, results in bad vibes among team members and doomed project.

Ok one more, you have been in same project/team/location for too long, project is good, team is good but stagnation is creeping in. This is one common problem with key resources in project, they work well and sometimes it proves to be negative for their learning curves. Technology or non technology, any job require people to re-skill themselves continuously. But business continuity and criticality being ground reality does not allow frequent resource movement.
It’s not something which can't be managed, we always handle such problems when people decides to part way with organization so why not before. Any team or manager too dependent on a particular resource(s) should always evaluate "what if" scenarios. At the same time, proactive initiative from management to provide growth opportunities to critical resources in timely manner will not only bring more productivity, enriched skill pool but also employee satisfaction and loyalty towards team/unit/organization. Irrespective of hierarchy, employee feels valued when organization care about their career and personal growth.

Though it’s unrelated to above context but I will still add “number matters but only collectively”, what it implies you don’t want to kill your best “Hen” to get all golden egg at one shot’ . Enrich your hens and golden egg will continue to come for long time.

Last but not the least sometimes personal problems are carried over to work. It’s not unnatural to have influence from home on work and vice versa but when it happens very frequently and you are unable to segregate the environments and your responses then it’s time to change and change for good.
Finally we all have different lives and thought process to handle our own problems/issues/concerns and there is nothing which fits all hence don’t be afraid of change when your coffee tastes bitter because change is not that bad after all.

Monday, August 31, 2009

What’s with the cloud anyway?

What’s in the name?

Cloud” is popular metaphor for internet, added with computing it represents highly scalable distributed computing capacity available through World Wide Web.

What’s the buzz?

Availability of affordable high speed network bandwidth allowed access to massive computing resources and services from across the world. Traditional datacenters are costly to setup and even costlier to maintain. Businesses are unwilling to own and manage non core business resources. Data suggests the average enterprise spends $8 in maintenance for every $1 spent on new IT infrastructure. Cloud offers on demand infrastructure and software service with flexible pricing options (pay per use or subscription model). Key is don’t own, use what you want and pay what you use. Gartner predicts by 2011, early technology adaptor businesses will buy 40% of IT infrastructure as a Service.

Companies like Google and Amazon built massively scalable processing and storage infrastructure to support core business operation which they are now harnessing to provide on demand infrastructure and service scalability to other businesses. Interesting bandwidth uses by AWS (Amazon web services) already exceeds their core e-tailing services.

Design basics - Service Layers of cloud computing

  1. Software as a Service (SaaS)

Highest layer offers business functions software running in cloud as service to clients. Software deployed on cloud act as service which can be exploited by user as per need basis, this scenario is very different from traditional “software on desktop” approach. Saas frees user from software licensee purchase and installation and works on pay per use or flat subscription model. Actually everyone of us has used it in one or other way.

Most widely known commercial examples are Salesforce.com and Google apps, Zoho, Microsoft online services.

  1. Platform as a Service (PaaS)

This middle layer offers platform abstraction to clients. Widely known example Xen image from Amazon platform cloud, contains linux distro, web server and development environment such as Ruby or Perl. Other examples are

· Java Google Web Toolkit (Google App Engine)

· Python Django (Google App Engine)

· Ruby on Rails (Heroku)

· .NET (Azure Services Platform)

PaaS is generally constrained by service provider’s capabilities like Google only supported Python for app engine before recent addition of java. Microsoft Azure “.NET”. No cross platform so far.

  1. Infrastructure as a Service (IaaS)

Lowest layer offering standard storage and computing capabilities to client over the network. Hardware resources such as Storage, servers, routers and switches are pooled using virtualization technology to offer on demand hardware scalability. Commercial IaaS is synonym with Amazon due to highly successful EC2 (Elastic compute cloud) and S3 (Simple storage service) services.

Virtualization

Virtualization is most important technology in cloud computing arena, providing abstraction to applications and end users from actual physical resources. Virtualization allows resources (Storage, servers etc) to be treated as resource pool rather discrete systems thus allowing on demand allocation.

Virtualization types

  • OS virtualization
  • Platform Virtualization
  • Network Virtualization

The key benefits offered by virtualization are

  1. Higher resource utilization rate (server, storage)
  2. Resource consolidation
  3. Lower power usage and cost optimization
  4. Space optimization
  5. Disaster recovery and business continuation
  6. Reduced operation costs

Other important technologies include para virtualization i.e. enable single server to be treated as multiple, clustering which allow multiple servers to act as single server. Sophisticated file systems like ZFS which can support virtually unlimited storage.

Cloud options

  1. Public clouds are maintained by commercial vender with massive computing datacenters, running jobs from multiple clients on shared resources (Storage, servers etc). No exclusive client control on underlying infrastructure services.
  2. Private clouds are on demand infrastructure exclusively owned by single client with control over processing capacities and job scheduling.
  3. Hybrid clouds promises best of both worlds where client share some services while use other in exclusivity. Model promises a lot but add complexity of application distribution across the environment. Initial hybrid implementations are capable of handling stateless applications with no complex database and synchronization requirements.

The good

  1. Agility powered by highly scalable infrastructure environment
  2. Anywhere access to application aka location independence
  3. Abstract infrastructure complexities from developer, users and business
  4. Massive on demand scalability
  5. Optimal hardware utilization
  6. Cost advantage (Pay per use or flat subscription model)
  7. Eliminate over provisioning, don’t need it until actually use it
  8. Accelerated application development
  9. Functional offloading – using cloud only for specific resource intensive functions

And not so good

  1. Security and privacy have been biggest concern for the organization as application and data is physical located in shared environment.
  2. Control over environment as service also need to go through multiple maturity cycle to achieve industry accepted model.
  3. Disaster recovery – Though cloud offers best breed disaster recovery through underlying redundant infrastructure and multiple hosted environment but network disaster can pose threat system to availability.

Tuesday, June 23, 2009

RESTful Architecture Style

In the world full of design principles and architectures, every new architectural style promises to learn from the failures and fill gaps left open by previous ones. With so many architectural recommendations and frameworks around, what makes REST special and why it matters? Answer is simple it make “World wide web” or more popularly called “Internet” work and work like dream. Yes, our very own Internet is a loose implementation of REST principles or in other words internet is almost “RESTful”. Some people even go to the extent of calling REST as an extension of HTTP design.

REST stands for Representational State Transfer; this term was coined by Roy Fielding in 2000 in his doctoral dissertation. Roy is one of principal writer of hyper text transfer protocol (HTTP) specifications; dissertation is available online here <http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm>. REST concepts have been around for quite sometime but recent emergence of service based designs have put it under spotlight.  As Roy has summed

“[REST] scales well with large numbers of clients, enables transfer of data in streams of unlimited size and type, supports intermediaries (proxies and gateways) as data transformation and caching components, and concentrates the application state within the user agent components.”

We started with fewer and smaller organizations with even smaller IT solutions. Need for components communicating/transacting with each other were limited but as enterprises and businesses crossed geographical and demographical boundaries large scale integration became need of hour. Conventional architecture allowed less strict rules which resulted in disparate system which are hard to integrate without getting into low level implementation details. Designer around the world felt need for more standardized approach to how components should be modeled to make integration easier.

Service oriented architecture and web services gained momentum on promise of integrating heterogeneous systems with utmost ease. SOAP interfaces (web services) solved many integration issues but complexities of solutions sometime resulted in more problems than intended solutions. Even flexible WSDL interfaces could not achieve complete independence from underlying implementation. It’s almost impossible for two arbitrary designed components to work together without knowing interface conventions and transfer messages. REST tried to address issues related to non restricted architectural styles resulting into to component specific implementations.

As architectural style, REST defines set of rules which system should comply with to achieve “RESTfulness” and reap the benefits. Why it is more appropriately called style because REST is very abstract as it does not define too many low level details unlike conventional architectures thus rather than being a concrete design it represent a style for architecting services.  By defining rules at highest level REST tries to prevent low level component locking. In past decade internet amazed everyone with growth, for a business to be successful internet presence become absolute necessity. Internet can be termed as biggest integration project in the history of computers and its architecture supported speedy expansion really well. This helped in making REST case even stronger.

At the core of REST is stateless client server interactions using uniform interface using which representation of resource state is transferred to requester client. It radically shift design from being API centric to information/domain data centric. You must have heard a lot about interface based design as generally termed as “design to interface” where underlying implementations are abstracted from consumer clients. Pretty neat, but REST takes it to next level by keeping interface constant. Every piece of information offer same set of methods with single identification scheme. New services add new piece of information which can retrieved/manipulated using standard requests. Unlike to RPC style systems, where services are modeled around what they do. 

In formal terms REST defines objects as Resources, Resources are identified by single identifier scheme typically a URL or URI, implementing similar set of methods to support information manipulation and retrieval, thus each involved component understand each request and response. Each part of request complies with an interface. There are no adhoc messages. Conventional design proposes "do something" Vs REST's "make something so" though both result in similar outcome but focus moves from verb to noun.

GET - Retrieve information or state of resource

PUT - Update information or state of resource

POST - Add information

DETETE - Remove information

These methods are borrowed from HTTP, though REST is not tied with HTTP but as present most successful implementation of REST style, internet is based on HTTP so it has found way in to the REST acronyms. “GET” being most used method in web application to retrieve information is well suited for extending its role to RESTful designs. Basic idea is to implement these standard set of methods for each entity or resource to enable standard retrieval and manipulation across the heterogeneous, geographically separated systems.

Here is an example* to show conventional Vs REST implementation, An RPC application might define operations such as the following:

getUser()

addUser()

removeUser()

updateUser()

getLocation()

addLocation()

removeLocation()

updateLocation()

listUsers()

listLocations()

findLocation()

findUser()

Client code to access this application may look something like this:

exampleAppObject = new ExampleApp('example.com:1234')

exampleAppObject.removeUser('001')

With REST’s resource centric approach

http://example.com/users/

http://example.com/users/{user} (one for each user - where {user} is either the user name or the user id)

http://example.com/findUserForm

http://example.com/locations/

http://example.com/locations/{location} (one for each location - where {location} is the location name or the location id)

http://example.com/findLocationForm

Client code to access this application may look something like this:

userResource = new Resource('http://example.com/users/001')

userResource.delete()

Important point is resource representation is completely independent from state of resource. Client can request representation based on it own requirements. This approach enables multiple format consumption support at server end serving same resource in multiple formats (i.e. HTML, XML).  Its like enabling web GUI and web service interface for the resources.

Another important REST principle is statelessness of server, each transaction between client and server to be self contained so that server need not keep track of served client and consecutive requests from same client can be served by different server resources, making better use of available system resource and easier replacement of failed units. Imagine no more memory hogging client session, but this does not mean REST does not allow stateful design, state can be maintained at client end, in other words client is responsible for providing complete request including authentication info each time, on server session or no session, cookie no cookie, server provide same resource state to each client. This also enhances “cachebility” of resources to reduce network hopping.

Summing few advantages, offers better support for load balancing with implicit statelessness, high level of client server independence, does not need separate directory mechanism due to single resource identification scheme, layered design support. In real world, Amazon S3 and blogs are great example of RESTful service implementation.

With some much attention around REST, EE6 has proposed to include JSR311 -JAX-RS: The Java API for RESTful Web Services. This API will enable developers to rapidly build Web applications in Java that are characteristic of the best designed parts of the Web. This API promises to make RESTful web service development easier to abstract away low level details much like JAX-WS does for SOAP protocol.  

Though REST promises a lot, but as we will see more and more business implementation, actual value of investment will start appear so will the issues. As it proposes data centric approach thus there is a learning curve in thinking solutions in terms of resources and their state. It is easier to model simple read around REST but complex business scenarios yet to be tested, there are still lessons to be learnt and only time will tell whether all gaps are filled or should we start looking for another new approach.

Saturday, February 21, 2009

Good design is good business!

Ideas require favorable circumstances to flourish but facts can stand for themselves in all the situations, only difference being in adverse situations we start analyzing facts more closely. This statement is more relevant in today's scenarios when world economy is not painting very pleasant picture, client spending is shrinking, even out right good ideas are hard to sell. So what is the fact that can stand for itself even in such situation “Good design is good business".

Though design holds uttermost importance in any product lifecycle but here I am specifically referring to technical design of IT systems. So, why good technical design is good business?

Software services allowed businesses around the globe to expand beyond geographic and cultural boundaries, reaching to diverse customers in time and cost effective manner. But at the same time inability of IT solutions to respond to changing market place, completion and customers have forced otherwise successful businesses to shut down. Close analysis of any such successes and failures invariably points towards IT system design. In current economic scenario IT system design becomes even more critical for serving and served businesses.

Our organization serves customer all across the globe creating and maintaining client business support systems. For continuous business growth it requires organic (more business from existing client) as well inorganic (new client business) growth in client IT business. Moving up knowledge ladder of existing client's IT ecosystem provides vast opportunities for new business development and good solutions (read designs) created by us serves as steps in that ladder reaffirming client’s faith in our capabilities and delivery model. They are like Oscar trophies; next movie (project) can easily generate funding based on previous great work. So why good design makes sense for everyone involved?

In simple words, better design always translate into better code, making maintenance  and extension cost effective and easier to justify to anyone needing it. Thumb rule is good IT systems are self sustaining, making them favorite of higher up management and frankly who doesn’t like hassle free IT systems saving corporation millions spent on maintenance and failed attempts of extensions. Also, ability to scale provides business confidence to respond to changing market dynamics. Remember everyone want a piece of good cake i.e. interface with flexible, bug free (ok lets settle for almost bug free), stable system, scalable good design can help in giving each one a piece they deserve.

For serving businesses like our organization, good design is easier to code and providing most efficient resource and time utilization (if you include time and effort spent in coding (fixing) bugs for bad design). Good designs leaves scope only for small coding bugs, but save time spent on filling functionality gaps. Good system can be sold for extension even in bad times because they are built to scale with least cost, but other way (bad system) it is hard even in perfect times.

Finally in good times bad design may go unnoticed but in bad time good system (design) stands out so do the bad system and you would not want to be standing out with second one.

So let’s go over few simple rules for creating good design

  • Traditional development model (water fall etc) requires design to start after requirement phase but starting early with high level design uncovers relevant questions in early stage saving repeated cycles of communications and documentation between business and technical teams. Believe me knowing business better and first hand always result in better technical designs.
  • Design should justify each of participating technologies. Though it will be superficial to say technology decisions can be cost independent but key is to identify right technologies to use based on business requirements, evaluate various available options (products), quantify results, share with business, if not easy , won’t be impossible to get business sing your tune.
  • Always design to services and components, map requirements to components/services and interfaces and things will start falling in place, giving you right kind of questions to ask.
  • Don’t run to code, instead spent time on understanding requirement and creating right design else you will be spending more time filling functionality gaps later. Right design will ensure you are left with enough time to realize it in the best possible way.
  • Don’t reinvent wheel, use best practices and patterns to save time and standardize your solution.
  • A layer of abstraction solves hundred problems by making interface information clearer for each business component and consuming channel.
  • Business components design should not be too coupled with consumer channel, remember web is not the only channel, it’s just one of them, if not code, design for business components with multiple interfaces to ensure system is capable of further extension with least impact.
  • Functional and non functional are equally important, you cannot make system faster and scalar if you never created functional design for one. Believe me you will spend twice the time doing the same thing if you first code for functionality then for speed and scalability.
  • Share design thought with developers to ensure high level design sync with low level realities.
  • Always remember testing is not just a phase in development cycle it is equally important to designing or coding, bottom line if you can't find it, you can't fix it. Stress on allocating right amount of time, effort and tools for testing.

This all sound reasonable for new systems but what about existing ones, existing system enhancement and maintenance is bigger chunk of work we do for our clients. Existing system are great teacher for anyone willing to learn about what to do and what not to. There are numerous good and bad systems present in IT world giving practical examples of do and don’ts.

Redesigning any such system which is not build around good design practices is not an option most of the times but few techniques can still help in making future enhancements easier and less time/effort consuming.

  • Analyze new requirements and existing system; remember code and data can tell you what business document may not.
  • Create high level design with component contract specifications for new enhancements.
  • Analyze interfacing with existing system components and identify ones that are violating your expected contract terms.
  • Target these components for sandboxing. Create abstraction layer around such components to prevent design decencies with new design/code.

Lastly, no system is perfect but we can aim to reduce gap between real and prefect system to minimum by making good designs.

Friday, November 28, 2008

Evolution

Human killed only for hunger
he evolved,
he learnt new ways of creating life,
he learnt to connect with other humans parted by great distances,
he learnt to reach sky and talk to stars,
he learnt to predict catastrophes
and
he learnt killing for religion
he learnt killing for color
he learnt killing for language
he learnt killing kids and mothers
he learnt how to kill without feeling pain of killed
he learnt killing for reasons
he learnt killing for no reason
he learnt killing just for killing
he only learnt killing after all !
he evolved to be inhumane!

Shame we going back to, where we started from!

Tuesday, October 21, 2008

Quebec City is located approximate 255 Km away from Montreal. In fall season,colorful arrays of trees on both side of highway make journey quite pleasurable. Inside Quebec city you find yourself among old stoned building which is kind of nice if you are coming from new age city where sky scrapper are build with glass and steel. Prominent places to visit can be divided in to two part one inside walled city and other on the periphery outside the wall.

DSC01552 DSC01555
Way to Quebec City Quebec Parliament

Quebec parliament is humongous architecture built with stone, symbolizes glorious French culture. Interesting no Canadian flag is visible in this part of Canada for all obvious reasons. But this building score excellent on the scale of architecture and beauty.
Walled city is located on the top of mountain and give picturesque view of water front on one side and outer Quebec city on the other, this place has it all, mountain highs, waterfront (Saint Lawrence River), lovely surrounding, great heritage buildings.
Inside walled city external old house structures are maintained to recreate magic of old French times but insides have been converted into restaurants and shops to cater needs of 9 million visitor coming here every year. I would have loved to see street markets and shops recreating magic of old Quebec city but like everywhere commercialization has taken its toll on glorious past.In spite of all this, Quebec city is undoubtedly one of most beautiful cities I have been to.

DSC01621 DSC01606
View of Walled Quebec City Saint Lawrence River

Hotel Le Château Frontenac is built on a tall cape tall cape, is symbol of Old Quebec city, it overlooks Saint Lawrence Rive giving spectacular view of vast natural beauty around Quebec city. This hotel has not only been part of french historical moments but also been silent observer to the second world war strategy discussions between Winston Churchill and Franklin D. Roosevelt at Citadelle.

Picture 226 Picture 248
Hotel Le Château Frontenac Wall painting


In addition to beautiful structures, Quebec city also displays various forms of art to visitors, if you notice carefully its a wall painting not actual people in their balconies
If you have desire and strength to climb on mountain top then Quebec city provides you ample reasons to do so, every single step towards top gives you a unique view of Lawrence river, ships and city below. It was worth going to the top when I could capture these shots.

DSC01653 DSC01651
View from top of hill

This journey will always stay alive in my heart and soul. As I wrote in previous post, some cities has their own life, Quebec city is surely one of them.

Sunday, October 19, 2008

1984 – My take away

Geogre orwell's 1984 is not only a interesting but also very compelling read. Each finished page puts you in hours of thinking. Somewhere it says a good book is one which tells you things which you already know but in more systematic way so that you can collaborate your thoughts around it.1984 does not state any unobvious or unknown fact. But, instigate your thought process on patterns which have existed forever in our society.

Some books have their own life; they live through centuries, through vast geographic diversities, making sense for people living in Mathura to Manhattan equally. You don't have to read these books but instead they talk to you as you turn pages. I know few of them Fountainhead, The class and 1984, this list will surely grow as I get to read more books.

"1984" posses a fundamental question of power and its existence. Why war and enmity between nations exist and why it will exist forever. Book repeatedly states three unique rules of power, 1) Ignorance is strength 2) Freedom is slavery 3) War is peace. They are more like oxymoron but when you read through context its all start making all sense.

I will start with my favorite one "Freedom is slavery", As we live in society we follow rules and regulations and become part of the system. In other words we are slaves to norms instituted by society & government and in return government promises our freedom. Any individual not confirming to social rules can not ask for same guarantee to his freedom. By being part of society we loose our identity as individual and any crime against us is considered as crime against society thus inviting appropriate punishment by system or government. Or in other words, in any disintegrated society each individual is responsible for his own safety, by not confirming to any rule though free, but responsible to keep his freedom intent. Slavery to system is key to keep personal freedom.   

"War is peace" is an oxymoron but again equally relevant in current context, war with external forces is always used by governments around the globe to maintain peace inside the country. War provide required strong emotions to suppress all other issues of importance, when country is at war no one questions tax hikes, ration cuts or delay in otherwise development projects. Anyway what is greater than national pride to citizens. Over the years governments and rulers have used this to best of their interest, Correlating this with current situations around start making sense immediately. Billions getting spent in defeating external enemies while internal enemies like poverty, illiteracy and epidemics always stay undefeated due to unavailability of required funds. Governments are eager to put billions into weapons and bombs but funds always fall short for giving food to everyone. A good war is useful to everyone but poor. Rulers & government gets full hand on funds usage which would be questioned otherwise, middle class gets something to watch on TV in evening and feel proud about. Frankly speaking, all money spent on wars around is more than enough to eradicate all other vices like poverty, hunger, infant mortality. But again who really cares.

Last one is "ignorance is strength", this one is even more appropriate in context of our country. In addition to all other benefits education enable us to ask questions. It opens window of mind, it develops a sense reasoning behind all actions. We start asking questions to people in responsible positions. We expect accountability from other leaders while Ignorant masses provide unquestioned absolute power to rulers. For years, rulers denied right to education to all section of society to suppress this reasoning, accountability and transparency. Ignorance is surely an strength for rulers who want to avoid accountability and transparency .

This novel was written sometime in 1947 but equally contemporary to today's situation. History repeats itself and in social context it repeats more often. Years passed, centuries passed but society always remained divided in three sections High, middle and low. Rulers changed, name changed and way of ruling changed but even societies and countries created in the name of human equality could not remove these sections. Few from middle moved to high, few from high fallen to middle but low remained low, in darkness. Interesting, middle always took support of low pretending to fight for their cause, to reach higher but once they reached there they pushed low back to where they always were. Every time after bit of imbalance everything settled back in same format which existed forever.