#Public domain text?
15 messages in this thread
Frank —
I hope that you feel philosophical regularly! Your message made for very
interesting reading. Thanks.
One combination I'm currently considering is Microsoft Viewer (for text)
combined with Sequiter CodeBase (for database data). Of course, Viewer is a
Windows-only solution. And it doesn't help you with screen design if you want
to do anything interesting at all.
What I would like is a UI designer that is as easy as Toolbook for creating
applications with multiple forms/pages; a text storage and retrieval engine
with royalty-free full-text search built in like Viewer; and the database
capabilities of FoxPro, all in one package!
— Samuel
Samuel,
What is Viewer? Is it a _search_ and retrieval package, or are there
indexes prebuilt. Without prebuilt indexes, searching our largest database
(80meg) becomes a career at 153kb/sec. The package we're using now keeps only
info about whether a word is in a document or not, and then has to read each
document found by the boolean search to see if the words make a phrase. We
spend lots of time looking at the busy light <no grin>.
But yes, in general you've describe what I'd like to find.
Frank `
Frank:
What package are you using now, if I may ask?
Rich
Rich,
We license our software from a company called Laserworks, who publish a legal
disk for the state of Illinois. The user interface is very lawyer-friendly, and
we like it a lot, but the guy who had it written didn't believe that phrase
searches were necessary so they're an afterthought, and the user interface code
was designed when they were looking at having only 2 or 3 databases on the
disk, whereas we have a couple dozen, rising all the time, so a bunch of
if..elses are now case statements, and the whole thing needs to be
restructured.
We're buying a ROMware license (Nimbus) and having a front end written,
but I can't not notice that if this was a database situation we could do it
ourselves, no problem and a lot less money.
Frank
Frank:
>>I can't not notice that if this was a database situation we could do it
ourselves, no problem and a lot less money.<<
True! Since we have talked about it before, I know you're familiar with SGML
and its implications. One of its side benefits _can_ be the "databas-ation" of
text content. Have you looked around for retrieval/presentation packages that
utilize these features? The only two I've heard of that actually _do_ it with
SGML are Fulcrum and Dataware. But I would think that could make a lot of
difference in such text-heavy applications as law pubs.
Rich
Pardon my poking into your message stream, but I was browsing and found your
mention of SGML coding as a basis for database-like actions. I do that is a
system I've created using Sony's FTR software. One of their selling points is
the ability to handle fielded as well as text materials equally well in the
Windows environment with one package. I say, their "selling points" because
that's how they sell it. And it does work, but maintaining a database in that
structure proves to be cumbersom. We're going to do one of two things for the
future: abandone their software for the database aspects of our work, or write
some maintenance tools to make our lives easier. If you want to know more, or
chat on this subject, give me a shout. -Steve Shaffer
Steve:
I'd love to hear about your experiences. Is the SGML used by FTR compliant
with the standard? Do they provide you with tools to maintain the dtd's and
the text yourself, in order to make future production more efficient. How well
is the text component integrated with the "database" (which I assume means
structured) data?
Thanks for the input. – Rich
Rich, I have a lot to say, good and not-so-good, about Sony's FTR program.
Overall, it works and does what it's supposed to. It's just a bit cumber- some
to use and maintain, and the documentation is, well, "light" would be one way
to describe it. What I should do, and will if/when I get a bit of time, is
write an honest, critical review both for potential users such as you and for
Sony's benefit in future revision and updating of the program. I'll try to do
that, and upload it here for public consump- tion in the future. In the mean
time, if you or anyone else has specific questions you can call me at
1-510-582-6624 9-5 pacific time and I'll be happy to chat. Talk to you soon,
and see you in San Jose.
-Steve
Rich,
Those are the two that I know about as well.
I never got as far as talking price with Fulcrum, but the way they
danced around the issue left me with the belief that since I had to be
concerned with cost, I was talking to the wrong company.
Dataware is a real possibility. (And I have heard that their software
is actually Fulcrum based). If they get a little hungrier and we get a little
flusher, perhaps something can be worked out.
Frank
Frank:
Actually, Dataware has their own engine, and then acquired a license to Fulcrum
when they acquired Reference Technology last year. I understand they are
marketing both. And the RefTech package has some really neat-sounding
conversion stuff, using Avalanche's scanning OCR conversion to SGML marked-up
text, and so on. I'm hoping to talk to some of my old buddies from Boulder in
San Jose and find out what the skinny really is these days.
Best – Rich
Rich,
I never really saw the _Dataware_ full text engine in action. Their
image based stuff is really fine, but until they bought RefTech their heart
really wasn't in text. (I always assumed because they did such great page
images<g>)
I'm a little constrained talking about Reference Technology because we
are vendors to them, competitors in the data prep business, and did discuss
being customers at one time. At this late date I'm not always really sure what
I know under what nondisclosure agreement. (This of course applies to the
Cambridge end of the company too, but what's to hide about TIFF files<g>)
For our application, besides the cost, we have a real interest in
maintaining simplicity in our user interface, and according to my revered
business partner we couldn't sufficiently customize the interface.
Also, FWIW, as far as the lawyer's dream system, at last years SIGCAT,
I talked to some folks from OAG (Official Airline Guide) that had an aircraft
maintenance CDROM that actually took account of the differences from serial
number to serial number in aircraft production. This would be wonderful for
keeping track of what the law was on the date that something happened, rather
than just what it is now. Again, an idea for someone with a million bucks. (and
50 state marketing of a Federal disk to get the million back.) Even after the
NRE was paid, I'm not sure that New Hampshire or Vermont could even pay back
the on-going maintenance.
And another throwaway: We've looked into FastTag, and don't find it to
(usually) be worth the trouble. It takes some serious training time, on a
per-job basis, and you still have to have _people_ look at the document(s), and
fix Fast Tag's mistakes as well as the OCR's. For the same kind of effort, we
can have the proofreaders put in hints and then write some job- specific code
to do the tagging algorithmically.
I can see that the situation might be different if you had clean,
untagged electronic copy, from the customer or off-shore keying, or if you were
putting images on the disk and were willing to index on a dirty scan just to
get the right page image. However, the second case is dumb: If you come to us
in the first place we can deliver tagged copy for less than the cost of
offshore keying plus tagging in-house. I don't know how real the third case is:
We do get jobs that are being full text indexed with images being displayed,
but the ones that actually come to us have a higher accuracy requirement than a
dirty scan would give. It's possible that we simply don't see the ones that are
done from new 12 point laser printed copy that would OCR at 99.98% accuracy,
but when it's check-writing time we don't find people actually willing to live
with 99% accuracy.
Frank
Frank:
Thanks for the insight. There is no better opinion than that from the person
who has walked a mile (or 12) in real-world shoes.
Will you be attending SIGCAT again this year? I'm thinking about going over if
I can spring the time (and money).
Best – Rich
Rich,
In our experience SIGCAT was fun, but no way for us to make a living.
It was full of Government people who were either
1. Tire kicking
2. Happy to use us for free consulting before giving the whole job to someone
else or taking our example of equipment/software and bringing the whole job
inhouse.
3. Sounded good–told us they wanted stuff put into WordPerfect, and then
_after the fact_, said "And Oh yes, the pagination has to be preserved." (For
other vampires on the taxpayers carcass, WordPerfect will _NOT_ allow you to
make a 6000 character 2 column page)
Oh yeah. And the government always takes 120 days+, not 30, to pay.
I will admit they're better than most of the competition, but I just ran out of
nice things to say.
Frank
Frank,
Viewer is a Microsoft product. It's what they've used to build Cinemania, the
MS Developer's CD, and Encarta.
It DOES have full text indexing, using their own search engine, and it is
royalty-free. It's based on the Windows Help paradigm, where you have "topics"
containing text, bitmaps, hotspots, etc. Searching seems to be very fast
(looking at their demos).
Where it falls short is in UI support. If you need to do your own UI, you have
to write a DLL. And of course it has no database support.
— Samuel
The basic idea of how Windows Help works is described in most C compilers for
Windows as well. At least it is in the Borland docs.