Monday, February 3, 2014

Remote Dependencies, Convenience, Risk and Other Considerations for Operating Distributed Systems

One deeply held principle by experienced distributed system operators that I have worked with is that you should have no external dependencies to your software other than the ties to minimum requirements of the OS such as common system libraries, utilities, and the kernel of the base OS. This approach should enable recreating a distributed system deployment without any dependencies on the outside world. When something goes wrong, you should have control over your own destiny. Reliance on any external dependency that is managed or hosted by someone else introduces risk that something outside your system can affect your ability to restore and recreate the system any time you need to.
To use a simple metaphor, imagine your system is represented by Jenga blocks and it falls over as Jenga towers inevitably do. However, instead of being able to rebuild your tower you find out that a mandatory required component at the base of your tower is missing or unavailable now no matter what you cannot rebuild the tower exactly how it was before. Your new tower is going to behave differently in unexpected ways and you might topple over because you do not understand all the behaviors when using different building blocks combined in a different way.
Some of the original designers of the software deployment project for Cloud Foundry named BOSH (Mark LucovskyVadim SpiwakDerek Collison) embraced this principle and tried to create a prescriptive framework that encouraged this approach. They had experience managing large scale distributed systems at Google (the web services APIs). Kent Skaar also did similar for SaaS provider Zendesk. Given a software release that references specific versions of multiple software packages (known as a BOSH release), an instantiation of that release (a BOSH deployment) can be reconstructed at any time with the deployment configuration (a BOSH deployment manifest), the base OS images (the BOSH stemcells) and the software release (the BOSH packages and job templates for applying configuration). at any point in time, properly implemented BOSH releases of large scale distributed systems can be recreated without external dependencies. That means this holds true even when the internet is unavailable.
BOSH does give you the framework hooks to break out of this prescriptive principle and use external dependencies or at least external dependency formats if you choose to for convenience or other reasons. Dr Nic Williams recently implemented tooling to use apt packages instead of compiling from source. another example: some of the Pivotal big data software intentionally targets CentOS/RHEL only and therefore only ships rpm packages rather than compiling Hadoop. A guiding principle is that you should be mindful of the tradeoffs you are making of convenience vs risk and tying your release to only one OS distributor.
Examples of the tradeoffs:
  • relying on an externally hosted package manager like apt-get could affect the availability or correctness of that dependency when you need it most
  • relying on debian packages could prevent someone from using your release unmodified with a CentOS image
A recent real-world example demonstrated the risk of an external dependency changing unexpectedly. The coreos/etcdproject that Cloud Foundry is using for storing stateful configuration data for the new Cloud Foundry Health Manager codebase had one of the dependencies (goraft/raft) force push to master of their git repository that overwrote some git history required by git to work properly. This situation has limited the flexibility of some users to make code modifications on several previous releases of Cloud Foundry without some tedious intervention.
A common reaction when learning about Cloud Foundry BOSH is to question the prescriptive guidance to compile from source when commonly used distributed package management systems exist in the Linux distributions. My recommendation is to understand the tradeoffs involved and make the best choice for your situation. You should explicitly call out external dependencies if you have them in your system. When you tower inevitably falls over, know how to rebuild it.

75 comments:

Esteem Host said...

Web hosting India offers you a domain name and more space. The virtual market is the best way to directly interact with the target audience. Because of the large number of businesses provides web hosting services in India and abroad.

Steve Hawks said...

The attitude towards cloud computing is influenced by multiple factors acting as drivers and barriers for its adoption. This research paper aims at the reconceptualization and operationalization of widely accepted influencing factors on technology adoption in relation to cloud computing.

cloud computing training chennai

Arthur Mac said...

Understanding the latest concepts is possible only through contents like this. Thanks for sharing this page in here. It will be useful for my future projects as well. Keep blogging articles like this.


Hadoop Training Chennai

Mathew Stephen said...

The content published here was worth able to read and share. The aspect in which you have written the content is amazing. I have bookmarked this page for future use. Thanks for sharing this in here. Keep blogging content like this.

Software testing training in chennai | Software testing training | Automation testing courses in chennai

Jhon Abraham said...

To keep ourselves up to date with the current trend is not an easy task in IT. But we can, through quality and worth able content like this. Thanks for sharing this web page. Please write more articles like this in future.

Hadoop Course in Chennai
| Hadoop training institutes in chennai | Manual testing training in Chennai

akhil sharma said...

very good post ....i enjoyed it while reading
best valentine day poem
valentine day whatsapp status
valentine day message for facebook

Rahul Singh said...

great
happy valentine day
valentine day wishes for boyfriend
valentine day poems for girlfriend
valentines day message for wife
valentine day message for girlfriend

Vikyjohn said...

Thanks for the post.Hadoop is an open-source software for storing data and running applications on a hardware. It provides storage for any kind of data, enormous processing power and the ability to handle tasks. Hadoop changes the enterprise store, process, and analyze data.

Hadoop training in Chennai

Chandrika Madapudi said...

Excellent post!!! In this competitive market, customer relationship management plays a significant role in determining a business success. That too, cloud based CRM product offer more flexibility to business owners to main strong relationship with the consumers. Cloud Computing Training in Chennai

Mansoor said...

Really informative blog , also visit us for : nebosh courses in chennai | nebosh in chennai | nebosh course in chennai

saki said...

thanks for sharing your technical knowledge of article for user purpose. its more informative keep sharing.
Informatica Training in Chennai

Thanu Abi said...

This post is amazing.It is very useful for me.Important topic to explained very nice.thanks a lot for this information.

cloud computing training in chennai

Akshay Saini said...

Server Wala Data Centers is a Dedicated Server provider located in Jaipur(Rajasthan). We provide the same excelling Bare Metal Dedicated Server, Managed Dedicated Server. We offer real live 24/7/365 support via phones, email, and chat and we are constantly striving to be better. We offer tons of extra features that other hosts do not provide, plus the ability to host unlimited domains on most of our packages! We offer a 30 day money back guarantee on most plans, so try us and if you’re not satisfied you can cancel for a full refund.

Keerthi said...

Now it is known to me that articles is nothing but inspiring is everything to do something great. This is a great article for the people who want to come in freelancing.

Informatica Training in Chennai

John Snow said...

Can anyone recommend me a dedicated server hosting company, with cheap dedicated server hosting ?

Luzy Anvi said...

i enjoyed it while reading safety courses in chennai|Nebosh courses in chennai|IOSH courses in chennai|safety courses in chennai|Diploma in safety courses training institute in chennai

Fazill .A said...

Nice One :)

fire and safety courses in chennai

Priya said...

The cloud computing offers many benefits for your business that can reduce the cost of managing and maintaining the IT systems.
cloud computing training in chennai | cloud computing courses in chennai

Nasreen Basu said...

interesting piece of information, I had come to know about your web-page from my friend pramod, jaipur,i have read atleast eight posts of yours by now, and let me tell you, your blog gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanks a million once again, Regards Guidewire training in hyderabad

Durga Spplimited said...

safety course in chennai
safety courses in chennai
construction safety courses in chennai
Industrial safety courses in chennai
offshore safety courses in chennai
fire and safety courses in chennai
fire and safety course in chennai
nebosh courses in chennai
safety engineering courses in chennai
Industrial safety engineering courses in chennai
fire and safety engineering courses in chennai
Iosh course in chennai


Paul Miller said...

Informatica is a (ETL) tool Extraction, Transformation and Loading. It is used for building enterprises with data warehouses.Learn Informatica course in Chennai and it has a massive demand in the market.To know more visit our website.
Informatica institutes in Chennai | Best Informatica Training Center in Chennai

Andrew Son said...

wow!!This is very interesting and i have all your blogs.Keep on blogging about cloud.
Regards,
cloud training in Chennai | cloud computing training in chennai

Arthur Mac said...

Thank you for taking time to provide us some of the useful and exclusive information with us.
Informatica Training in Chennai | Informatica course in Chennai

Rajapriya R said...

nice and really helpful article to everyone... thanks for sharing

selenium training in chennai | selenium training institute in chennai | Android training in chennai | android training institute in chennai

Saradha Devi said...

great one

Best Selenium Training in Chennai | Android Training in Chennai | Java Training in chennai | Webdesigning Training in Chennai

Geetha Devi said...

Thank you for sharing
Best Selenium Training in Chennai |Selenium Training in Chennai |
Android Training in ChennaiBest Manual Testing Training in chennai

Rajapriya R said...

informative blog

selenium training in chennai | selenium training institute in chennai | Android training in chennai | android training institute in chennai

Geetha Devi said...

Nice post Best Selenium Training in Chennai |Selenium Training in Chennai |
Android Training in ChennaiBest Manual Testing Training in chennai

Rajapriya R said...

informative blog

java training in chennai | java training institute in chennai | java j2ee training in chennai | java j2ee training institute in chennai

Priya R said...

Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
salesforce training in chennai | salesforce training institute in chennai

Haxie 100 said...

Happy New Year 2017 Wishes
Happy New Year 2017 Informations
Happy New Year 2017 Quotes Check These Outt

Ranjit Ram said...

Want to know about Veterans Day,which is a day to celebrate the great officers of the US army,just checkout my blog Happy Veterans Day and stay updated.

Amirtha rao said...

Nowadays, most of the businesses rely on cloud based CRM tool to power their business process. They want to access the business from anywhere and anytime. In such scenarios, salesforce CRM will ensure massive advantage to the business owners.Cloud Computing Training in Chennai

Saradha Devi said...

nice one keep sharing
Best Selenium Training Center in chennai | Best Automation Testing Training in Chennai

geethu said...

Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas.
Salesforce Training in Chennai | Salesforce Training Institute in Chennai

Priya R said...

The strategy you have posted on this technology helped me to get into the next level and had lot of information in it. The python programming language is very popular and most widely used.
Python Training in Chennai | Python Course in Chennai

Anu Priya said...

Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing.
Regards,
AWS Training | AWS Training in Chennai

Trick Junction said...

Valentine Week 2017

Sujitha Reddy said...

Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing. Android Nougat 7.1 www.facebook.com log in mobile best love status whatsapp dp images download hd

Sujitha Reddy said...

Nice articles are shared on behalf of you, really good to check these urls also you may see some of the best collection on
Good Mrng Image lovely good morning images good Morning Inspirational Quotes good morning friends

Rasool Bevi said...

Excellent post on iOS mobile apps development!!! The future of mobile application development is on positive note. You can make most it by having in-depth knowledge on mobile application development platform and other stunning features. iOS Training in Chennai | iOS Training Institutes in Chennai

Abinaya V said...

Our current course offerings are on Android, Azure, Google Cloud, Red Hat, MongoDB, Chef, Docker, Big Data, DevOps, Cloud Computing Security & Analytics and IoT. We are adding more courses that will delve deeper into these subjects.
Android Training in Chennai | Android Training in Chennai |
Android Training in Chennai |
Android Training in Chennai |

Santhosh raj said...

Big data analytics training
MBA in event management
MBA in marketing management
learn core java online
online aptitude training

sivanesan said...

The best thing is that your blog really informative thanks for your great information!
erp in chennai

Amirtha rao said...

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.
Regards,
Salesforce Training in Chennai|Salesforce Training

Nuy Nurhasanah said...

a given article is very interesting and very useful for me, thank you very much admin and pardon me permission to share the article here :

Obat gabagen
Obat radang lambung alami
Obat ginjal bocor tradisional

Tina Kenard said...

Nowadays, most of the businesses rely on cloud based CRM tool to power their business process. They want to access the business from anywhere and anytime. In such scenarios, salesforce CRM will ensure massive advantage to the business owners.Cloud Computing Training in Chennai

Rasool Bevi said...

The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling. Hadoop Training in Chennai | Big Data Training in Chennai

anitha a said...

You absolutely come with great well written articles. Thanks a lot for sharing your blog. You have posted a trust worthy blog keep sharing
Film editing course in chennai

Siva Seo said...

Thank you for this valuable information. I have got some important suggestions from it. I'm working in Brave Technologies Private Limited. web designing company chennai

Amirtha rao said...

This was worth reading article, java is no doubt the most powerful programming languages yet more demanding as well. J2EE Training in Chennai | JAVA Training in Chennai

Logavani G said...

really informative blog you have been posted. thank you for sharing such kind of an interesting article. keep on sharing.
selenium training in chennai

Rasool Bevi said...

Loved your blog
Regards
DOT NET Training in Chennai

Nandhini said...

Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. Java Training in Chennai

Rasool Bevi said...

I went through your blog and it’s totally awesome. Keep on updating your site with such informative post. If possible please include rss feed for your blog. Web design training in Chennai | Web designing course in Chennai | Best web designing course in Chennai

Mathew Stephen said...

Great and impressive article!! Got to learn and know more about web development. To know more refer create website for much more unique ideas.PHP Training in Chennai | Best PHP training in Chennai

meghanasmiley03 said...

Very helpful tips information. Thanks for sharing. For jobs visit vmware jobs in hyderabad

Logavani G said...

really nice blog. it will be really helpful to the peoples who are all working under the technology. so thank you for sharing this blog. keep on sharing this kind of useful blog.
selenium training in chennai

Amirtha rao said...

The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling. Hadoop Training in Chennai | Big Data Training in Chennai

Bay Max said...

I went through your blog and it’s totally awesome. Keep on updating your site with such informative post. If possible please include rss feed for your blog. Web design training in Chennai | Web designing course in Chennai | Best web designing course in Chennai

Rajesh Kumar K said...

Blog really informative you posted. Thank you for sharing this interesting article. Continue to share. selenium jobs for fresher’s in hyderabad

akhil varma said...

Impressive post..!!!
I can learn so many interesting or informative post through this blog that is helpful for us...Amazing work you have to do on this blog..Keep sharing much more post...Can you please share etl testing jobs in hyderabad.

vini vinitha said...

Nice Blog.Thank you for Sharing. I'm working in brave technologies private limited.we are leading cloud erp software solution in chennai

Eneng Nur said...

given article is very helpful and very useful for my admin, and pardon me permission to share articles here hopefully helped :

Cara mengatasi bopeng
Obat herbal otot jantung lemah
Cara menyembuhkan paru paru basah

Imperial Voyages Tour Operators said...

imperial voyages travel and tour operators in delhi we provide all affordable travel and tour packages at lowest price, if you need tour packages in india please visit our website. tour operators in delhi

Nuy Nurhasanah said...

articles as a very interesting and rewarding once, and pardon me permission to share articles here hopefully helpful and useful :

Cara menyembuhkan amandel bengkak
Cara mengatasi bab berdarah
Cara mengatasi telinga berdengung

Ramya Krishnan said...

Great Post, Thanks for sharing your valuable article. it is very impressive to read the article.
Web Designing Training institute in chennai | Web Designing Training institute in velachery

Krishna Veni said...

Good one.. thanks for sharing your views and ideas..


Android Training in chennai | Software Testing Training in chennai

Imperial Voyages Tour Operators said...

imperial voyages travel and tour packages at cheapest price we provide all india tour packages at cheap and best price. tour operators in delhi

Sanjana E said...


Remote dependencies and convenience of operating distributed systems should have control over destiny and other external dependency.
java training

Vicky Tawde said...

Thanks for sharing this information with us, i always like to read these articles, cool..!
24JobsExam website
Check 24JobsExam links


Visit here
SSC CGL
UPSC Recruitment
Employment opportunities
Govt Jobs
Result
Admit cards
24Jobsexam
Careers News
Exams Notification
Read more

Deni Sumarga said...

I really love your blog.I wanted to thank you for this great read!! cara mengatasi buah zakar bengkak dan nyeriObat Fistula Ani

nuy herbal said...

may be useful for all, helpful article once and pardon me permission to share also here

Obat infeksi saluran kencing
Cara menyembuhkan penyakit kuning
Cara mengatasi ginjal bocor

chandhu said...

Thanks for sharing such a wonderful information with helpful content...keep updating..CCNA Training Institute in Chennai | CCNA Training Institute in Velachery.

deeskha said...

your blog contain very useful information and good points were stated in the blog which are very helpful one.
ISTQB Certification Training in Chennai | No.1 ISTQB Exam Center in Chennai | Best ISTQB Certification Exams in Chennai