Website Suggestions archives

How to "suck up" and "process" all that is at this site and in the archives??

Posted by RocketHealer Jim++ on October 30, 1998 at 08:41:20:

I was reading one of the archives this AM and I got to thinkig about how to gather and to process and to distill all that is here into something even better and easily searchable. To separate the wheat from the chaff. Yes, humans do that, to an extent, with a mental computer more powerful than anything man has been able to build so far, as they read. But I'm thinking something perhaps more high-tech, perhaps using artificial intelligence, pattern-matching, data-base technologies, whatever that would benefit all who wanted to "tap into" the storehouse without spending days and weeks reading and processing things close but off topic for them.

Of course, perhaps that is exactly what people need to do in order to get well. Could I/we do what I'm suggesting, it might be simplifying the process too much, eliminating the "required search/study effort" factor of the recovery.

There was a site search function here, but apparently it took the host computer to its knees, so it had to be disabled.

Web spiders etc. can index the site, so you can find specific pages, but that's not really what I have in mind.

Much effort has gone into this site and making the archives etc. I would not propose anything that would require any extra work from Walt, etc.

I'm looking for something like a web spider to "suck up" all that is here and "process" it into something much greater and more easily accessable by the masses than the parts now here. Ideas??

No, I'm *NOT* expecting the AMA to fund this research and information dissimination effort! :-)

RocketHealer Jim++


Follow Ups:


That is a non-trivial proposal you make

Posted by Pete R on October 30, 1998 at 11:19:20:

In Reply to: How to "suck up" and "process" all that is at this site and in the archives?? posted by RocketHealer Jim++ on October 30, 1998 at 08:41:20:

Hello RJ. I build things and must agree with you that there
is a need for such a product. In many areas we suffer from
information overload and a way to sort it out electronically
would be most useful. It could make someone a lot of money
too. Consider an email device that would get rid of the
stuff you were going to just delete anyway?

But thinking about actually "building" such a device
(i.e. the software algorithm) is what discourages me.
If I had a team of say 10 people, working on this for 1
year, I'm not sure how valuable the results would be at the
end of the contract. What I'm getting at is that a project
of this magnitude is most likely going to be performed by
a company the size of microsoft, or some start-up that
has some excited venture capitalists. It would be big work.
Also, do you have a basis of where you would start? Current
neural networks are used primarily for pattern recognition.
There is no magic there. It may be some rule-based system,
with the user entering his personal search criterion, and
then filter and reorganize the database on those
selections. And such a system will of course make mistakes.
In summary, it will take more time than most of us have,
but you propose a very interesting project.

Regards. Pete Reinhard




Index to Posts

Posted by Sharon on October 30, 1998 at 14:18:29:

In Reply to: How to "suck up" and "process" all that is at this site and in the archives?? posted by RocketHealer Jim++ on October 30, 1998 at 08:41:20:

I don't know went on behind the scenes, but Steve Shapiro has done an index for posts by Peter D'Adamo's on the Eat Right 4 Your Type website. And Peter just posted that he has finished the index to the Ask Dr. D'Adamo page. I don't know if it was all manual labor or what? You could email Steve or look at his site and Peters. You can get to Steve's site through www.dadamo.com


Follow Ups:


Re: Index to Posts is just that

Posted by RocketHealer Jim++ on October 30, 1998 at 16:10:44:

In Reply to: Index to Posts posted by Sharon on October 30, 1998 at 14:18:29:

Thanks, I looked at what they have available. That is really nice as far as it goes. It looks to me like that is simply a chronological linked index to the posts. Unfortunately, the subject lines (there and here!) really do not do justice to what's in the individual posts. Some may contain great insight, others just chatty stuff.

What I'm after is some way to get a hold of the good "content" and make that easily accessable. The closest software product that I've identified are CasePoint and CBR Express, from Inference. But they don't put pricing info on their on-line literature. Usually if you have to ask about pricing, then you CANNOT afford it!

So my idea is mostly just an idea for now, pending finding just the right software to accomplish it. I'm an old FORTRAN programmer. I can really crunch the numbers, but there is no way I'm going to write something "textual" and "intelligent" like this in my spare time!

RocketHealer Jim++



Follow Ups:

Yup, it's expensive!

Posted by RocketHealer Jim++ on October 30, 1998 at 16:18:57:

In Reply to: Re: Index to Posts is just that posted by RocketHealer Jim++ on October 30, 1998 at 16:10:44:

I was right. I emailed them for prices and found out: The cost for 1 licence for the search and retrieval tool (CASEPOINT) is $1500 plus maintance. The cost for the authoring tool (CBR Express) is $7000 per seat.

I mentioned what I had in mind doing and they said: "We
do have customers that have developed medical knowledge bases and deployed them on their intranets. There is quite a bit of cost involved in creating a knowledge base."

I'd say that's an understatment!

So.......I guess I can buy a *lot* of books for that price.

Perhaps my next idea will be a much better(and perhaps doable) one :-)

RocketHealer Jim++



Re: How to "suck up" and "process" all that is at this site and in the archives??

Posted by Walt Stoll on October 31, 1998 at 11:25:30:

In Reply to: How to "suck up" and "process" all that is at this site and in the archives?? posted by RocketHealer Jim++ on October 30, 1998 at 08:41:20:

Dear RocketHealer Jim,

My son, Bill, has suggested to me that I look at the archives and write a "generic" introduction for each of the most commonly used subjects. I keep saying that I will do so but seem to keep procrastinating.

Also, Bill told me (when the search feature went down) that it would get back up in not too long a time. Now that it has been about 6 months, perhaps a few encouraging words from the BB participants might help.

He told me a week or so ago that BCN had greatly enlarged their capacity and that was exactly what they said they had to do to get the search feature back.

Perhaps, together, we can make this easier. I DO think it is important for peop[le to put in some of their own effort. HOWEVER, the effort presently needed is "above & beyond".

Thanks for bringing it up again.

Walt



Follow Ups:


Above and Beyond the call!

Posted by RocketHealer Jim++ on October 31, 1998 at 16:26:13:

In Reply to: Re: How to "suck up" and "process" all that is at this site and in the archives?? posted by Walt Stoll on October 31, 1998 at 11:25:30:

I have looked at the existing archives some. Having been here a while, it is now time for me to go thru them some more, this time starting with my much greater understanding than I had earlier. I downloaded several of them today to my hard disk so I can look thru them at leisure without tieing up my phone line and running up hourly charges. Netscape will display them off-line from the .html files just like it does on-line, only much faster! Of course that does tie up some of my disk space, but I can think of few better uses for disk space than this health research effort.

I think it is good to say, "go read the archives first" but starting from ground zero, they don't do that much for a person. There is some basic understanding that is required before simply reading the archives will do a person much good, IMHO.

Your idea for some intros is probably good...but when is that likely to happen? If you are like me, probably never. There is always something more important to do than that.

In looking at the archives, I thought (briefly) about trying to cut/paste or simply write down "Walt's Rules" (for different conditions and in general) as I went thru them. After I thought about that briefly, I realized what a herculean task that would be for a lay-person like me (even a Rocket Scientist! :-). It's an idea, but not that doable for an individual. It probably would be better accomplished by someone knowledgeable starting with today's posts and some sort of indexing system. (like indexing a book)

Your son Bill mentioned to me recently that you might have to eliminate some of the archives due to disk space limitations of your ISP. That sounds in the opposite direction of what you just said about their expanding their capabilities. If anything, I'd personally like to see the archives not only retained but perhaps even subdivided further into more topics, and perhaps hyperlinked to the "Walt's Rules" that apply to the particular condition. But that sounds like a lot more work for someone, and it does not suggest how that work might be paid for. Cost-Benefits usually rule. :-)

Thanks, (It's all terrific just exactly as it is!)
RocketHealer Jim++



Self-Organizing Maps

Posted by Aaron Wieland on November 01, 1998 at 02:31:37:

In Reply to: How to "suck up" and "process" all that is at this site and in the archives?? posted by RocketHealer Jim++ on October 30, 1998 at 08:41:20:

This is probably a long shot, but you may be interested in the link provided below. WEBSOM is an adaptation of Kohonen Self-Organizing Maps to the automatic cataloguing of Internet messages. I've experimented a little with the demo; it's interesting, but the navigation feels somewhat odd. I don't know whether clustering posts in a two-dimensional space, using keywords, would be helpful or confusing to newcomers.

Cheers,
-- Aaron Wieland



Follow Ups:

Self-Organizing Maps look WONDERFUL!

Posted by RocketHealer Jim++ on November 01, 1998 at 11:16:53:

In Reply to: Self-Organizing Maps posted by Aaron Wieland on November 01, 1998 at 02:31:37:

These look wonderful to me. I've emailed them about the practical application of what they've developed and demonstrated. I'm not expecting to find it in a $39.95 shrink-wrapped software package to run on my PC for a few years yet.

Thanks so much for pointing this out. It's exactly what I had in mind. It was just not common knowledge that this had already been done. But nearly every "great" idea I ever had either turned out to have already been done, or was accomplished within 10 years after I thought of it.

Jim



Return to Dr Stoll Home Page

Post a Message

Main Archives Page

More Website Suggestions archives