Home » Forums » User Support » Suggestion

Grid Applications...

Posted March 28th, 2007 by larry
Hello Sugree, It's been awhile since I posted, but university has been busy. An area I've been working in lately has been one you like very much...grid computing. I have been wondering how difficult it would be to commercialize grid applications and make it more mainstream. I have read all your grid posts on this site and have seen a few examples either from big vendors or some highly specialized batch-submission type stuff. While this is all good, IMO it will not help make grid applications mainstream. So, I went looking around to see what I could find. For middleware, I found Condor, Globus, Legion, GridBus, and Unicore...to name a few of the more popular and stable projects. The problem with these projects, as they pertain to the web, is that they merely hook existing computers together and then create all the cool stuff grid computing can do. In other words, if you don't have 1000 extra PCs lying around to link together or a cray supercomputer at your disposal, these systems won't do much for you. So, I looked at Desktop Grid computing solutions. The most famous being Seti@Home and all the other projects using the same software. Seti@Home runs the BOINC platform, which is great because this software is more web oriented. It's a client-server structure that has millions of clients running on the PCs of those who wish to participate in the project. This is very different from the grid middleware I discussed earlier because you don't need to have 1000 extra PCs or a cray supercomputer lying around to make a grid. The users themselves, who install the grid client, create the grid to provide functionality to the system. BOINC is open source which is good. It is near impossible to configure for a commercial application, which is bad. I've looked at XtremGrid, but it appears a bit outdated. However, with it you can create a pseudo p2p network on your grid. A hybrid, which is cool. Do you know of any other, application-building-friendly, open source grid systems that can be used as a starting point for building grid apps on the web? Something I may have missed here? cheers, larry
‹ manufacturer for digital satellite receiver DM500s in China Nenest - A Web Application Framework for Non-Developers! ›
  • 1959 reads
  • at

desktop grid framework

On March 30th, 2007 sugree says:

I'm glad to hear from you about desktop grid. There are so many implementation of desktop grid framework. Anyway, it might need some work to make it usable.

  1. Alchemi Led by one of the most famous guy in HPC, Rajkumar Buyya.
  2. OpenUCI Developed by my junior
  3. JXTA I have just heard about Harvest project in ANSCSE2007 which implemented P2P grid using JXTA.

Most of this kind of things are just research topics of master degree students. It might not perfectly work as you expect.

  • reply
  • th

Alchemi and JXTA...

On March 30th, 2007 larry says:
I've looked into Alchemi and JXTA. They're interesting, but I have concerns. Alchemi is only available for Windows. I suppose that's okay, but long term...who knows. I looked at JXTA a few weeks back and it looked dead. The impressive project list is old (years) and the projects themselves seem abandoned. Am I wrong on this? I believe JXTA is platform independent as well...which is an advantage. As for the Harvest project, this is along the lines of a framework I was imagining. However, I cannot find a project page...only papers! Do you know if there is a project page? How come all the cool projects are closed off...and then they write papers about it!? Anyway, what do you think about Alchemi and JXTA? Myself, JXTA appears to offer more web-service oriented stuff such as IM with jabber and the p2p/grid-hybrid-topology you mentioned, along with the old projects on the JXTA page as starting points. Alchemi seems to be in use more than the JXTA framework and seems better supported. Is this accurate?
  • reply
  • at

jxta

On March 31st, 2007 sugree says:
Actually, I can't find any information about `Harvest` too. However, I personally prefer to work with JXTA more than Alchemi. Sadly, JXTA seems out-date. It would be nice if someone develop opensource P2P framework for all of us. This is a good topic for you.
  • reply
  • th

gt4

On April 2nd, 2007 sugree says:
Absolutely! I have worked with Globus Toolkit since it was GT1 until now GT4. As its name, GT is just a toolkit aka libraries. You have to develop your own application yourself. There is no built-in framework. The original design aimed to standardize set of protocols and their APIs. However, the goal has changed to merge with WSRF as requested by many vendors. As of now, GT4 offers set of libraries for developing grid services as an alternative to traditional web services. The most significant feature of grid services should be the built-in stateful service.
  • reply
  • th

gt4 and sakai...

On April 2nd, 2007 larry says:
I'm a bit confused. You said "There is no built-in framework", however, from what I have read it is meant to be a framework to build upon. Could you please explain to me how (theoretcally) I would set up a grid and allow users to interact with it. Here is what I now think. This is for gt4. 1. I set up a VO (virtual organization) on my server with the gt4 software. 2. I build an application on this VO for others to access. For example, I could use Sakai for collaboration. I choose Sakai because globus mentions them on their website under the 'Grid Software' and then 'Collaboration' links. 3. Users would download a VO grid client from my site (although theoretically it could be from globus.org itself or any gt4 grid website), which is actually a generic gt4 client, and receive a certificate authority to use my grid services from me. 4. They then would access my services using the client for authentication. If they have not received a certificate authority from me for my site, they cannot use my services. Am I right with the above? If not, could you please show me where my thinking is wrong. As I've said, I have not used globus...just read alot and this is my idea on how the system works. Thanks for looking at this.
  • reply
  • at

framework and library

On April 2nd, 2007 sugree says:
GT4 consists of a large set of APIs and additional tools for constructing grid environment and grid application. GT4 is not such a framework. That means you have to develop everything from scratch without any restrictions. Framework, in my point of view, is something that provides basic functionalities of the system and let us extend more functionalities ourselves conforming to the framework itself. For example, PHP is a library and Drupal is a framework. You can do anything by PHP but less in Drupal. In contrast, your work will be minimized since Drupal provides content framework, authentication, authorization and many more. You might think grid services as a framework sometimes but actually, it does not. I don't recommend to use generic GT4 client. You should write one yourself to fit your need. It should be generated automatically when you defined server part.
  • reply
  • th

globus and unicore...

On April 2nd, 2007 larry says:
I thank you for the clarification. As usual with much open source software, documentation and explanations on usage are lacking. Do you know about UNICORE at unicore.eu? What is the difference between gt4 and UNICORE? You see, I am looking for a framework. Much like your excellent php/drupal example, I do not want to rebuild a functioning grid framework if I don't have to...it's a lot of unnecessary work and I'm not a specialist...yet. So, I want to find a grid framework and develop from there. My thinking is much like a p2p framework. A p2p framework only exists because of the users that have downloaded and installed the available client. The client talks to other clients that speak the same protocol and...boom...you have a p2p framework. Grid currently works in two ways, cluster-like architecture where machines are connected to one another, normally at one location (but not always) and form a virtual supercomputer. The software installed to make this happen is not user friendly. The second is a desktop-grid architecture. This resembles p2p in topology, but the client only talks to the server...not other clients on the network. This is the Seti/Boinc server architecture. I am seeking a grid framework similar to Seti and Boinc seems to be the only thing available, however, it's lacking in the ability to allow peers to discover and "talk" to each other. Any ideas?
  • reply
  • at

unicore

On April 2nd, 2007 sugree says:
UNICORE is designed to be wrapper for all existing grid middleware including Globus. Some parts of them are open source. I'm not sure about its license by some projects used UNICORE for so long, e.g., research projects in Japan. If you don't mind, JXTA seems to be the most complete p2p framework and it is cross-platform.
  • reply
  • th

Ok...but what is JXTA?

On April 2nd, 2007 larry says:
Since I seem to have not properly understood everything else, could you please tell me exactly what JXTA is and does? I think JXTA is a ready made framework that builds a p2p network. I have a few problems with this...1. It's not a grid framework...it's p2p. But, I could work around that and try to gridify it later. 2. I would like to make the network closed with authentication for security reasons. I don't want porn or pirated stuff on my network. Anyone else who uses the JXTA client would be able to access the network by simply having a generic JXTA client. I'm learning all about protocols and I currently lack the protocol abilities to engineer a JXTA client to use my own personal homemade protocol. 3. JXTA appears outdated...but not dead. I could live with this and develop it myself if I chose it. It's funny you mention JXTA. I was visiting Sun and saw their grid engine. Do you know anything about the sun grid engine? I appreciate all your help. It appears everything I read about these grid projects, and I've read alot, is misunderstood. It's almost like they don't want you to get involved.
  • reply
  • at

Globus...

On April 1st, 2007 larry says:
I have taken a look at Globus at Globus.org and I have an idea, however I have never used it...I have only read a bunch of papers on it. Do you know anything about Globus or have any experiences with it?
  • reply
  • at

SGE and JXTA

On April 3rd, 2007 sugree says:
JXTA will let you establish a mesh network on the fly and provides you necessary namespaces and routing for transmitting and receiving any kind of messages to/from any nodes. Security is up to you. JXTA will take care all routing automatically. In JXTA, you don't need to worry about its protocol. It works. Another possible solution is to use OpenUCI which is baesd on .NET. It was developed by my junior at university as his engineering project. Sun Grid Engine or SGE is a batch scheduler for cluster and it could be used in the low-level subsystem under Globus job manager. Don't confuse about its name. SGE is not an engine for the grid but it could be a part of them.
  • reply
  • th

Post new comment