#Xbase database software
26 messages in this thread
Ben,
I have a CD-ROM writer available at work too. Right now, I do not have
any data for use in testing. I just wondered if somone out there had ever
tried to develop a CD-ROM product using Xbase software.
Thanks anyway, Phil
Philip,
I have been useing FoxPro V2.5 to manage a 420 Mb, 490,000 record database of
aviation safety data on one-off CDs. I tried Paradox for DOS first but it
inflated the files to about twice the size of .DBF files. I have the .DBF and
.CDX files on the CD and run FP from the hard disk. Just don't forget to
specify a path when writing a file to disk.
When I had my CD drive hooked directly to a SCSI card the performance was
nearly as good as from the HD, sometimes actually better. I now have my drive
hooked to the paralell port on my notebook for portability which slows things
up considerably. It's still acceptable though.
Some complex searches take awhile from either HD or CD. But it's relatively
simple to output a subset to a new file and work on them.
My biggest problem now is that the FoxPro Developers Kit does not include the
RQBE screen. And because my databases have so many fields and every user wants
to do unique searches there is no way that canned searches are adequate. I
found a third-party add-on called FoxFire in the FoxForum which can do ad hoc
searches, but the license fee is a little pricey for my application. If anyone
can tell me how to include the RQBE in a runtime version then FoxPro will be
perfect for my CD application.
Let me know if you have any questions. Kathy Cochrane at the One-Off Shop
Mid-South produces my CDs and I'm sure can answer any questions about that end
of the process.
Regards,
John
John,
What is RQBE? Does that mean Query By Example? I am not a FoxBase user
so I am not familiar with their terminology.
You have a pretty large database. Is it all in a single file or are
there several files that are cross-linked? If the are multiple files, does
accessing multiple files slow down response time?
Thanks for the information. Phil
Phillip,
RQBE is Relational Query By Example. Which is another way of saying menu
driven searches.
Linked searches with FoxPro on CD are pretty slow. All my databases are flat
files. Which in most cases are no problem with all the available space on CD.
As an experiment I linked a couple of files on CD and did a very simple single
field search which took 115 seconds to come back. The same search on CD but
not linked to another file took 0.5 seconds. Of course a lot of the
performance depends on database design. But I would certainly avoid relational
files if at all possible.
Regards,
John
John:
I hope I'm not being redundant by pointing out that relational databases grew
up on hard disks, where it was actually efficient to profligate in the number
of disk accesses required to get, compare, and get another piece of data over
and over again. The key to great performanc eon cDROM is to reduce the number
of disk seeks required.
Which is the long way 'round of saying your experience in your experiment was
typical and expected. Thanks for sharing with us!
Best – Rich Bowers
Hi Rich.
While I don't disagree with your specific points about relational databases and
CD-ROMs, I don't think I do agree with the implication that relational
databases shouldn't be used with CD-ROMs.
Rather, from my *limited* experience with CD-ROMs, I haven't seen anything that
reflects a lot of in-depth experience with relational tables. Most of what
I've seen so far (and that's not a whole lot) seems to have been developed
around text-indexing/search engines by folks who really seem to understand that
technology — but maybe not database technology. And, I'm just not sure that
for some applications, database technology might be a much better fit. At
least I'm not going to strike it off at this point in my CD-ROM learning curve.
What ya think?
Bill
Bill:
Thanks for your observations! I didn't mean to imply that one shouldn't use
RDB's with CDROM, only that there were limitations challenges:
1) If one simply carries the application across from the mag disk to the CD
environment, there will be performance problems because of the way most
(assuming I haven't seen it all) database software utilizes the disk.
2) The text-oriented database world has re-engineered their access
methodologies to create a CD-specific retrieval tool. This may also be
possible – certainly desirable – for the relational world as well.
Relational has a great deal to offer and has not been used effectively in other
media yet. Maybe the concept can transfer to new media with some new or
revised applications. In the meantime, a number of members here have discussed
their use of various RDB packages on CD. In short, go for it!
Best – Rich
Rich let me throw out a couple of other observations.
While, I've been hacking with database software for years, I'm still a
CD-ROM novice (but learning a lot from you and everyone else here
every day!). So, I'm really interested in reactions to these
observations.
First, I see two general types of data that can be used to describe
something that we want to understand: text data and attribute data.
For example, information on movies consist of descriptions and
critiques (text data) and attributes like title, year, type,
rating… etc.
While text data generally have little structure, attribute data do.
Therefore IMO, the optimum method for storing each type of data, and
the optimum techinques for searching and retrieving specific data
from each type, differ. For example, IMO, attribute data, because of
its structure should be stored in tables (database relations with
rows and columns) and managed with database technology. That's what
the database community has been optimizing for years — by utilizing
keys and powerful indexing technology, relating associated data in
different tables, etc. And, they have been concerned about minimizing
disk accesses too, since even on a fast hard drive, disk access time
is the biggest performance inhibitor.
OTOH, the typical database management system doesn't lend itself to
text data management. And, IMO, here's where the full-text-search
community has developed applicable technology. While I'm still
learning about this technology it seems very, very, powerful too.
Now, here's my "rub." It appears that the database-technology folks,
in general, have ignored CD-ROM media and left CD-ROM database
management to the full-text-search-technology folks. And, as a
result, I see full-text-search technology being used to search and
retrieve data based on its attributes — where I would think database
technology would provide a much more optimum solution.
One of my favorite quotes is:
"If the only tool that you have is a hammer, then every problem
becomes a nail."
Maybe it fits here. What ya think?
Bill
Bill:
I think your quote and, in large part, your perceptions, are right
on. Many software developers and publishers have been looking for
simple ways to produce discs and often over-look the needs of
customers. It has always been my bias that as much effort as
possible should be accounted for in the _creation_ of the product,
removing as much burden as possible from the end-user. However,
those with text solutions don't want to complicate their lives with
specialized retrieval of "attribute" data, and vice versa.
Part of our current problem is overcoming some rather simplistic views
of data in the past. We have not given enough attention to the
nature of different types of data, and their unique needs for optimum
handling. If you can create some solutions that will improve the
ultimate access to data, you may have a winner!
Best – Rich Bowers
> Many software developers and publishers…often over-look the needs of
customers <
Rich, from the little I've seen, I couldn't agree more. I developed a
personal FoxPro application several years ago to help me select
movies to rent. When, for example, I want a "good" western movie, it
will print me a listing of all the western movies in my database that
I haven't seen whose average of several critics ratings is equal to
or greater than three stars. And the list that it produces contains
the movies' attributes and is sorted by year (newest first) within
the average critics rating (best first).
The movie store folks think I'm crazy when I show up with my list. <g>
But, I don't waste time browsing through shelves and shelves of
movies in whatever order they decided to display them. And, I don't
bring home many duds.
So when I read about Cinemania, I said wonderful — 19,000 movies!
(Obviously my movies database is quite, quite limited.) But what a
disappointment. IMO, it's essentially "book technology" on a CD-ROM
with some fancy graphic gizmos. The authors assumed that their
"customers" already know the title of the movie that they want to
research, and set up the CD like Maltin's books. I know it will let
you specify filters, but you can't even get a list on the screen that
contains the movies' attributes — much less print one. And, you
can't sort anything.
The "Movies' Titles" are the "answers" that I, the customer, am
looking for — not my search and retrieval keys. I can retrieve
movie ratings by title using any of my movie books! And, I don't
even need a computer. Anyway, one guy's opinion.
> If you can create some solutions that will improve the ultimate access to
data, you may have a winner! <
Know anyone who wants to hire me? I'm experienced and available. <g>
Bill
Bill:
>>Know anyone who wants to hire me? I'm experienced and available.<<
Rich,
I see what you and Bill are saying, but I would suggest another reason
for the bias in CDROMs toward full text–The full text tools will do the
attribute job, whereas using the attribute tools to do the full text job just
doesn't work.
For a related problem, check out the bibliographic database people.
They have a job that 'should' be right up the Xbase alley. Actually the MARC
format is incredibly complex, and a useful 'content' catalog of even a private
collection makes one those warehouse or accounts payable apps look pretty
trivial.
Mr. Cobb did not have title changes, pseudonyms, books republished under the
authors real name, books republished as by the real author and the pseudonym,
second editions with some but not all the same authors, and so forth and so on
in mind when he laid out those pretty rules. OTOH, full text can handle that
kind of stuff even though it's pretty ugly.
Frank
Frank,
I understand what you said about the complexity of the MARC format, but
would full text necessarily be any better?
Let's take the movie database as an example. Suppose I want to search
for movies by name of an actor (e.g. William Boyd) or by name of a character in
the story (e.g. Hopalong Cassidy). In a full text database you could get a lot
of false drops if you cannot limit the search to a specific field. For example
if we search for (John & Wayne) we could retrieve retrieve a movie staring Jill
St. John and Wayne Newton, if they ever appeared in the same movie.
Maybe the way to solve this problem is to use SGML to delimit the
various components of the full text record. What do you think?
Phil
I wonder how much the FoxPro indexing method improves performance overall
versus a dBASE 4 style index? Have you compared the two at all?
Albert,
I've been wondering the same thing. If someone wants to loan me a copy of the
dBase (or other) .EXE file I will be more than happy to try it on my CD-R. But
unless the program can use the FoxPro compact structural index (.CDX) we won't
know what kind of performance is possible with indexed searches.
BTW, I have tried Alpha4 and R&R Report Righter on CD without success. I
suggest that anyone wanting to try another DBMS program on CD first try
accessing some .DBF files on a write protected floppy before spending the $250
for a trial CD-R.
But to answer your question, from what I know of the FoxPro technology I have a
hunch that it will outperform other Xbase software at least on CD. But have no
actual experience with others.
Like you I'm interested in other people's experience with Xbase and CD.
Regards,
John
After I wrote the message I began reading about index performance in my
(SoftC/Greenleaf database library). And sure enough the CDX files from FoxPro
are currently the fastest. The nice thing about the library you can mix any
type of indexes with any type of files. Currently I'm using dBASE 4 indexes on
dBASE 3 files. I should try out the FoxPro indexes and see what happens.
Have you started any threads in the FOX forum or dBASE forum on their
performance? (RE: dbf files on CD-ROM)
That's good to know, thanks.
The Fox and DB forums are a little over my head sometimes so I try to stay
around here or the Avsig most of the time. It's nice to have some idea what
the conversation is about. <g>
Regards,
John
John:
>>BTW, I have tried Alpha4 and R&R Report Righter on CD without success.<<
What went wrong? How did it fail?
Thanks – Rich Bowers
Rich,
Neither of them would open a database. I'm not sure what happened
with R&R, but Alpha tech support says that A4 must be able to write
to the same directory as the data is in. So a basic test for CD
capability is to try reading data on a write protected floppy. If
that is possible you have a good chance that it will work with a CD.
Then the question becomes performance.
John:
I was wondering if the read-only environment might not have been a
problem. And this is one of the reasons that much of commercial
database software can't be picked up and transferred directly to a
read-only medium. So I'm always interested in the things that _do_
indeed work.
Thanks for sharing with us!
Rich Bowers
Rich,
Fox seems to create all of it's temp files in the program directory
while A4 immediately creates a couple of trash files in the data
directory. Also when in Fox if you save a query or something you
have to remember to include a path or it gives you an error message.
Rich,
Are you saving the messages in this thread? I think that the
information being presented here has some long term value.
Phil
Hi Rich.
> the information being presented here has some long term value. <
I agree. But, I'm surprised that I haven't seen this topic discussed in depth
before.
But then, the computer-typsetting folks didn't converse very well with the
computer-word-processing folks (and vise virsa) for years. But, today, the
typesetter's bench has met the typewriter in many super computer applications.
Bill
> from what I know about FoxPro technology, I have a hunch that it will
outperform other xBase software at least on CD. <
John, I suspect you're right. I also suspect that it will significantly
outperform the full-text-search engines for retrieving CD-ROM data based on the
data's attributes. (See my note in this thread to Rich).
I've been a FoxBase/FoxPro hacker for years, and it is lightning fast — for
what it's designed to do.
Bill
> from what I know about FoxPro technology, I have a hunch that it will
outperform other xBase software at least on CD. <
Perhaps. The May 11 issue of PC Mag looked at the current databases
and noted in a sidebar that the newest version of dBASE IV is more or
less equal to Fox in speed now.
Also, note that you can now get similar technology as the Fox
bit-optimized query technology, in a C database library called
CODEBASE 5.0 (in case you want to code in C instead of Fox for your
app).
With regard to the comments about the suitability of traditional
relational database software for CD applications, I would agree with
a comment made in this thread: reducing disk accesses are exactly
what database software attempts to do already. What's good for the
goose (hard disk databases) should be good for the gander (CD rom
databases).
SoftC/Greenleaf (GO PCVENB) also supports Foxpro databases and CDX indexes.
Its very similar to Codebase.