Skip to main content

Need For a Simple Distributed Computing Tools

I have been working on a few hobby project these days. while going through,I came across two key challenges.

Challenge One

"The Need to tap in to parallel processing power of all the resources one might have in a simple way"

Assume the scenario of a person who is in a need to jump start a research activity/ some computationally intensive task, which needs a large computing power & resources. And he or she is in such a situation that they want to be able in include resources in much more dynamic way & is on a Mobile or Adhoc network where anything can happen. At the very same time i don't want to ignore the point that these days even our mobile devices have turned so powerful (a typical arm cpu of Samsung s4 will churn out 49 Gigaflops). And the overall system should be simple to use understand & operate. I came across a thess work Self organised software agents where the agent programs are designed in such a way that they understands the environment & behaves accodringly

Challenge Two

Flexibility to code ones solution in a known language & that it can be executed in massively parallel way. 
A simple solution found was to allow the algorithm to be build as blocks of programs. each block does a single job & uses a secondary scripting to define how the blocks of program be executed each block will be placed on a dependency graph, where the program starts from its start block and travels to stop block, which is as simple as defining a flow chart. To achieve parallelism each block can tell its either its a sequential block or a parallel block. Each block can issued new jobs which will be be en-queued in to the graph & executed

So the end solution is two tools in which 

one is software agents & the other one is a tool to generate or script the job for agent

when is comes to distributed computing two key factors controlling its performance is 

1. location of data or information
2. location of the program or algorithm which has to process the data

This is being resolved as program are written in blocks or small program chuncks
A global & local data pools



This was the story behind part for Acompute.

Comments

Popular posts from this blog

Going OpenSource ... Recently I have been thinking, I have finished several projects , but whats my contribution to the public cuz most of  the works have been for clients... So gonna start uploading some of the nightly builds over there

TechMeChennai IGovern Project

Recently we (tech me chennai group ) happend have a disscussion on grvernace system the following was the key statement, " Ban all religious & community parties & NGOs(RSS, VHP, NDF etc etc ) that doesn't represent the Indians as a whole from governance system & place a continuous evaluation electoral system for all the political persons or leaders, that includes A Performance Log only then these ppls will do good & be good Share & comment to discuss ‪#‎ RevIndia‬ " So then and there we decided to have a system of continuous evaluation for leaders at least we can design a protoype model for this. The challanges are   Being Unbiased & fail proof being reliable & live  simple & interactive accurate & available We need a technology system that can make its effort to keep track of factors in basic oaths Oath of office: I, <name>, do swear in the name of God/solemnly affirm that I will bear true faith and al...

Working With Kerala Police ;State Crime Records Bureau

Thanks to Mr. Habeeb Khan - Last few days i have been working with KSPD(Kerala State police department)  for my project .I should thank to my brother for helping me to get over there : he Introduced me to SP Jayakumar . who is the Cheif over there. things got moved swiftly. Got an Insight view about how things are beign done . I Also found that Crime Bureau in India have a Special wing which perform the near similar tasks of my project but of-course offline and manual . well its a good bussiness idea 3:) Update  Today my web-crawler have been built and its being tested to incorporate LDA algorithm for topic detection which i belive will give me relavent pages of interest My deadl line is 19th of this month : And IM running Really short in time : datacollection part is really getting tough as , the department will most probably hand over me with hard copy of data