Skip to main content

Need For a Simple Distributed Computing Tools

I have been working on a few hobby project these days. while going through,I came across two key challenges.

Challenge One

"The Need to tap in to parallel processing power of all the resources one might have in a simple way"

Assume the scenario of a person who is in a need to jump start a research activity/ some computationally intensive task, which needs a large computing power & resources. And he or she is in such a situation that they want to be able in include resources in much more dynamic way & is on a Mobile or Adhoc network where anything can happen. At the very same time i don't want to ignore the point that these days even our mobile devices have turned so powerful (a typical arm cpu of Samsung s4 will churn out 49 Gigaflops). And the overall system should be simple to use understand & operate. I came across a thess work Self organised software agents where the agent programs are designed in such a way that they understands the environment & behaves accodringly

Challenge Two

Flexibility to code ones solution in a known language & that it can be executed in massively parallel way. 
A simple solution found was to allow the algorithm to be build as blocks of programs. each block does a single job & uses a secondary scripting to define how the blocks of program be executed each block will be placed on a dependency graph, where the program starts from its start block and travels to stop block, which is as simple as defining a flow chart. To achieve parallelism each block can tell its either its a sequential block or a parallel block. Each block can issued new jobs which will be be en-queued in to the graph & executed

So the end solution is two tools in which 

one is software agents & the other one is a tool to generate or script the job for agent

when is comes to distributed computing two key factors controlling its performance is 

1. location of data or information
2. location of the program or algorithm which has to process the data

This is being resolved as program are written in blocks or small program chuncks
A global & local data pools



This was the story behind part for Acompute.

Comments

Popular posts from this blog

Working With Kerala Police ;State Crime Records Bureau

Thanks to Mr. Habeeb Khan - Last few days i have been working with KSPD(Kerala State police department)  for my project .I should thank to my brother for helping me to get over there : he Introduced me to SP Jayakumar . who is the Cheif over there. things got moved swiftly. Got an Insight view about how things are beign done . I Also found that Crime Bureau in India have a Special wing which perform the near similar tasks of my project but of-course offline and manual . well its a good bussiness idea 3:) Update  Today my web-crawler have been built and its being tested to incorporate LDA algorithm for topic detection which i belive will give me relavent pages of interest My deadl line is 19th of this month : And IM running Really short in time : datacollection part is really getting tough as , the department will most probably hand over me with hard copy of data