CDROM for Library
32 messages in this thread
Philip,
What is Library of Congress doing to transfer the paper (and crumbling)
books that our National heritage to CD-ROM discs? Can you tell us what the
plans, schedules, and prices are, and how to purchase?
I had heard that you are also the store house for many works done in
foreign countries. For example, the entire library of the research unit of
Hitler's Germany that successfully converted coal to gasoline, is in the
U.S. somewhere. Do you have it? I've heard that both you and the library
at Texas A&M has it.
How do we get it, or other such? Especially on CD-ROM?
Thanks!
Jan
Jan,
Man-oh-man! You don't ask for much, do you.
First of all, LC does have a Preservation Division which is doing
research into ways of keeping acid-based paper from deteriorating. The
only thing to do is to treat the paper with an alkaline that will
neutralize the acid in the paper. So far, the only treatment method that
works on whole books uses di-ethyl zinc, a gas that can explode
spontaneously. There have been attempts to take the di-ethyl zinc process
commercial, but it has not been successful because of the risks of
explosion.
The American Memory project has been trying to put some of the local
history materials on to CD-ROM, but it has been slow and expensive. I work
on putting machine-readable files such as the bibliographic catalog on
CD-ROM, and that can be difficult enough. When you talk about converting
text and illustrations to machine-readable form and putting them on CD-ROM,
it gets obscenely expensive. It might be great for you to have a copy of
Hitler's secret files on CD-ROM, but how many copies could LC sell? How
much would we have to charge per copy to recover our costs? Big bucks, or
BIG BIG BUCKS?
Finally, have you any idea how long CDs last? Every one seems to
think that they will last indefinitely, but they have not been around long
enough to evaluate their long term durability and readability. My hunch is
that they have only about a ten year useful life.
Phil
Phil,
Sorry late response.
You are the authority–I can only guess. I would say that all paper books
have a limited life, albeit perhaps very long for the better paper. That is
not the problem, the problem is that "database locked up in a sea of ink".
Accessability!
I know you have budget problems and it's going to take some innovative
ideas to get from paper to electronic format. I would imagine that it
could be easily funded by taking the most popular works, porting them to
CD-ROM then selling millions to the public. I'm sure there are many titles
that would fit in that category. If you priced the CD right, say $29.95 or
so (retail with allowances for distributors), you could reasonably expect
sales in the range of 50,000 per year on the good titles. This might give
you the cash flow to do the slow movers.
On the last point, 3M will guarantee their discs for a substantial (100
year?) life on the silver, and several people guarantee gold for much
longer. As the medium becomes for used, I'm sure life will go up. In
fact, the only current enemy, oxidiation, could be eliminated by air tight
sealing.
Will be interested in your response!
Rgds
Jan
Jan, >> the problem is that "database locked up in a sea of ink".
Accessability!<<
You have hit on a topic that is near and dear to the heart of Dr.
James H. Billington, the Librarian of Congress. Ever since he came to LC,
he has been talking about how to "get the champagne out of the bottle." By
this he means, how to make all of the information stored in LC available to
the public and to the scholarly community. If you have any suggestions for
him, you should write to him directly.
As I have already said, converting books or newspapers to
machine-readable form is very expensive. Whether it is scanned or
keyboarded, the text must be proof read to insure that no errors have been
created in the conversion. A related question is just what do we convert?
American history is probably the most popular subject that is present in
the LC collection. However, CD-ROMs on American history have already been
published and are being sold for about $100.
Anything that LC would distribute on CD-ROM would be in the public
domain, i.e. not copyrighted. If the CD-ROM is really popular, a private
company could copy it and undersell us. LC has already lost sales of our
bibliographic authority products to companies who have bought our files and
then undersold us. Under these circumstances, LC would probably not be
able to recover our cost, and would have to rely on Congress, or someone
else, to fund the effort.
What you are asking for is not impossible, merely difficult. You
should understand that the technical problems are the least difficult ones.
[More]
[Continued]
>>On the last point, 3M will guarantee their discs for a substantial (100
year?) life on the silver, and several people guarantee gold for much
longer.<<
I am not an authority on the durability of CD-ROM. I do have some
concerns about it though. Are the discs that 3M guarantees CD-audio or
CD-ROM? A sour note or two in an audio recording is a lot less noticeable
than an unreadable sector in a CD-ROM database. Secondly, the recording
layer in a CD-ROM is embedded in polycarbonate plastic. I could be wrong,
but I think that this type of plastic tends to deteriorate over time. As
part of the detioration process, the plastic changes color and becomes less
transparent. At what point does the color change interfere with the
reading of the recording layer?
Does any one out there have an answer to these questions?
Phil
Philip —
On CD-ROM durability …
I don't have any test data, but I know Kodak is not predicting a useful
life for their Photo CD disks. They are saying they should have a life
comparable to color negatives, at least.
John Spragens
Palo Alto, CA
Actually, I have seen predictions by Kodak reps (who may have been speaking
out of turn) that their CDs would last 25 years or more. At this point,
though, it's all really guessing. A minimum of 10 years is a safe bet,
though.
There are some other issues, as well. Some discs are made better than
others, and would have a longer life expectancy. Some discs are better
cared for than others, with the same result.
Kathy <<– The One-Off CD Shop MidSouth, Inc.
Kathy —
The 25 years figure would be in line with the color negatives statement.
As you say, it makes a difference how well you store them. Same goes for
color film. (Dark, cool, with moderate humidity is best for film.
Apparently for the Photo CDs, too.)
John
How long do color negatives last? I found some glass plate negatives in a
friend's basement dated around 1900 (his grandfather owned a major camera
shop). They still looked good enough to use. Jim
Jim —
Glass plate negatives probably are black and white — based on silver which
can last a very long time if stored carefully. The negs you found probably
*could* be used — and might be a lot of fun to have prints from.
Color negatives are based on dyes, which fade. The amount of fading
depends on the way they're stored. And, because some dyes fade faster than
others, the quality of the picture can deteriorate long before you reach
the point of having no picture at all. Look at a color print that's been
in strong sun a good while. The yellows probably will be all gone. The
magentas are likely to be seriously faded and the cyan less faded — so the
picture will have a very blue look to it.
If you want to find out more from people who've done serious study of the
technical end of photography, GO PHOTOFORUM and ask there. Kodak, Ilford
and Fuji all have people hanging around, and there are others with equal or
better experience.
John Spragens
Palo Alto, CA
Phil,
Please E-Mail me the good doctor's address. I will be happy to send him
some thoughts by mail.
Re: What to convert: Start with history, documents, photos, etc. of which
the LC must have the ultimate collection. Either invite some private
companies to co-produce (I'm sure that Bill Gates would JUMP at the
chance!), or do it on your own, but set up a pricing structure that allows
you to sell to/through distribution. There are many very good distributors
around, Compton's comes to mind (they're selling between 20,000 to 30,000
discs per week). I'm equally sure that many of the pressing houses will
probably extend credit for the work and accept payment after discs are sold
(public service minded souls, that is). Then price the discs so the retail
is low, say $29.95. You could probably command a wholesale/distributor's
price of $10 or so and should be able to sell 10-30,000 discs per title per
month (assuming you really get your teeth into this, go after schools,
libraries, dealers, etc). Starting with one title could give you a
positive cash flow of $100,000 per month, that would increment as new discs
are put on line. Now you've got the money to do many more, and make yet
more money (not for profit, of course, but to finance the further
conversion).
I am sure that others will have slightly different views of this, but there
is a wealth of talent that you can draw on here in the Forum.
The 3M folk are in the forum frequently–they would be the ones to advise
you on life of disc. Remember, however, that the disc master is made of
nickel and not subject to deterioration – just press more discs one hundred
years from now!
As to private companies undercutting you, they only will, or can, if you
leave them room. Price the disc either so they can earn a margin by buying
it from you and re-selling, or price low so they can't compete. You've got
the storehouse of data that is of value. No one in the world could produce
for less cost than you.
(cont)
Jan,
Actully, disc masters are made of silver, not nickel, by some companies.
The stampers, that are mirror images of the "father" disc, are frequently
nickel. Disc Manufacturing, Inc. (DMI) saves the masters, but replaces the
stampers after a certain period of use. This is over-simplified (I omitted
the step of making the "glass master" or "mother" disc), but may at least
show something about the process of mass-replicated discs.
One-Off CDs are made with an entirely different process, however. No glass
masters, no stampers. We just "burn" individual "blanks" that are
pre-formed using polycarbonate substrate, an organic dye data layer, 24K
gold metalized layer, and lacquer over the metal to protect it from
scratches and oxidation.
Kathy <<– The One-Off CD Shop MidSouth, Inc.
Kathy,
Sorry, you are correct–I was referring to the "stamper".
Not to pick nits, but isn't gold less susceptable to oxidation than the
laquer?
Rgds
Jan
Jan,
Why not pick nits? I was <G>.
Gold will eventually tarnish, although it isn't as much of a problem as
with other metals. The lacquer also protects it from being rubbed or
scratched off. Have you ever looked at a CD with a strong light behind it,
holding your fingers between the light and the disc? You can actually see
your fingers through the metallized disc. The metal is only a few microns
thick, like the gold on an astronaut's helmet. It's so thin it's
translucent, but it is still an excellent reflector.
Kathy
Kathy,
Well, yes, now that you mention it, when I was on the moon, I held my hand
up to shield my eyes from the sun, and I did see the silouete through the
gold on my visor! (joking of course, I wasn't really on the moon, I was
with Jean Luc on Eraumeus).
The metal is put there with "splutter" technology. It is supposed to be
one molecule thick.
The same technique was used on a pair of gold earrings I gave a girlfriend
when I was a kid. The green came our REAL fast!
Rgds
Jan
Kathy,
I have bitten my tongue, and bidded my time, but I can't let your comment
pass without challenging it. You said, "Gold will eventually tarnish,
although it isn't as much of a problem as with other metals."
Gold does not tarnish. It is chemically inactive. That means it does not
"actively" combine with other elements, most notably oxygen, or, in other
words, it does not tarnish.
The gold used on a writable CD-ROM is pure gold, or 24k. It is as
chemicaly pure as modern technology can make it. It has to be to work in
both the application (splutter), and reflection (non tarnishing).
Aluminium, on the other hand readily oxidizes. If it oxidizes as a crystal
you have ruby or sapphire (depending on the metal impurities present). If
boron is present, you get jade.
When you said that gold will eventually tarnish, you must have been
thinking of gold jewelry. However, in that case, it is not the gold that
is tarnishing (oxidizing), but the other metals in the alloy, usually
copper.
Now that I've gotten that off my chest, I feel better. Unless, of course,
that by your choice of the word "eventually" you meant a few billion years.
Rgds
Jan
Jan Schwenk wrote: "Gold does not tarnish. It is chemically inactive…
(etc.)
Geeze, go easy on the science, Jan. It took me 25 years to convince my wife
that gold DOES tarnish, that going very green very fast is a sure sign of
*quality* gold and that her friends' gold doesn't go green because they
just have cheap husbands. Next you'll be telling her that "rhine"stones
aren't diamonds that come from Germany instead of South Africa!
Jan,
Well, in geological time…. In fact, I was remembering comments from
geology professors, who do have a slightly different view of time than most
people. From a practical standpoint, you are correct. But scientifically
speaking, no material is absolutely, 100% pure, the 24K label
notwithstanding. But I'll stop quibbling if you will <GRIN>
Kathy
(cont from previous)
The LC is, of course, our heritage. The vast majority of citizens don't
have access. This is a way to give it!
Also, I am equally sure there are any number of students at colleges,
universities and even high schools that would willingly provide the labor
for free in exchange for the chance to do something for their country and
their school. Let the school earn a little from the sale to help support
the school's budgets. Now you have millions of willing, capable
assistants. There is nothing to stop you!
Is there?
Rgds
Jan
Phil,
It seems that much of what you folks (and the Smithsonian) have
done so far is primarily image oriented. It also seems to me that you've
still got a lot of opportuinity there: certainly a CD of page images is not
as useful as a full text searchable database, but it beats heck out of a
stack of yellowing paper locked in a warehouse. (FWIW, up here in the
boonies (Northern New England) Dartmouth is still running around
microfilming the small town weeklies preserved in the small town libraries,
and quite pleased with itself thank you). Since this stuff is of interest
almost solely to historians, I'm afraid the money does have to come from
that kind of source. But at thr right price I think you have the
opportunity to sell a whole lot more copies (every college in the country
instead of only univerieties).
As for text, I think you'll need to make deals with private
companies for access. We do scanning for a living, so I know _very_ well
<smile> just how much it costs. OTOH, I also know how hard it can be to
locate originals. You can solve that problem, and get at least your cosls
covered, and/or force the price down to where there's a copy in the public
library of every county seat in the country.
Frank
Frank,
>> certainly a CD of page images is not as useful
as a full text searchable database, but it beats heck out of a stack of
yellowing paper locked in a warehouse.<<
LC has tried this approach, and it was less than a resounding
success. It was call the Optical Disc Project, and used the old 12 inch
optical disc format instead of the CD yellow (or red) book format. The
problem was that in order to find anything, the page images had to be
indexed with key words. This required a lot of keyboarding. If all you
are going to do is make an image of the page, you might just as well
microfilm it. It's a lot cheaper. BTW, LC is microfilming a lot of the
periodicals in its collection. These periodicals are usually bound, and
have to be unbound before filming.
Right now, all the CD-ROM products that LC sells are
machine-readable (i.e. ASCII) files.
I thing you are right about getting help from private industry, but
"it takes a long spoon to sup with the Devil." If the material is at all
marketable, a private company will want exclusive rights to its
distribution. This may go against the general right of public access. It's
a tough problem.
Phil
Philip,
<<have you any idea how long CDs last? Every one seems to think that they
will last indefinitely, but they have not been around long enough to
evaluate their long term durability and readability. My hunch is that they
have only about a ten year useful life.
>>
PMJI, but one of the lovely things about CD-ROM is that when you get
nervous about the physical integrity of a disc, you simply make another
copy from that disc. The data remains identical, unlike on analog magnetic
media which deteriorate and slightly degrade their signal with each
playing. When someone has information they consider critical, I advise
making two (or more) copies of the same one-off, and storing one off-site
in a safe deposit box or the like. If something happens to the play-time
copy, simply make a backup from the stored one and restart the clock.
Kathy <<– The One-Off CD Shop MidSouth, Inc.
From the comments I've seen from others particularly interested in archival
storage of data (ie: 100 year minimum), it seems that a carefully cared-for
CD-ROM disc will likely outlast the technology.
That is, it really isn't so much how long the CD-ROM disc will last as
whether there will be anything around to read them in 20 years.
Jim,
The best part of CD-ROM is that it is digital, and consequently every copy
is an "original". As new technology comes into play, it will be a simple
task to transfer to.
Rgds
Jan
{Yes, I agree about the "original" quality of CD-ROM/digital information,
Jan. The discussion I was talking about apparently concerned very
high-level archivists — Library of Congress, etc. Apparently they wanted
something that would last at least 100 years (talking about gold vs. silver
CD-ROM, glass vs. plastic and such) and the question came up about the
reader technology.
Seemed to me, as it apparently does to you, that it would be simple enough
to keep transfering it to whatever new technology comes along to replace
what is available. For some reason that never came up in the conversation.
Cheers, Jan!
Jim,
Good talking to you again!
I agree, if the technology allows the transfer to digital format, then it
would be simple to transfer to the next newer technology.
The point is, however (ref the Library of Congress thread that started
this) that they don't have the funds to put the books on CD-ROM to begin
with.
Who knows what new technology will come down the pike? If you've watched
"Third Generation", Cmdr. Data can read and store a book simply by flipping
the pages at light speed.
Should we wait for Data?
Rgds
Jan
>>Congress thread that started this) that they don't have the funds to put
the books on CD-ROM to begin with.<<
You just hit the nail on the thumb!
The major expense of producing a text-based CD-ROM is converting
data to machine-readable form, and editing and proofing it. This costs
about ten times as much as all the rest of the project.
Phil
Very good point, Jim. So take good care of your CD player & computer, too
<grin>. However, the CD technology is already 10 years old, and ISO9660
dates back to 1988, so there is a certain longetivity to this format. The
standards folks are trying to keep the format backward compatible. It may
be like trying to play an old 78 rpm recording on a new turn table (are
there such things as new turntables since CDs are replacing LPs?). If
worse comes to worse, when the format changes too radically you could
re-master a disc from the old one onto whatever replaces it.
Kathy <<– The One-Off CD Shop MidSouth, Inc.
Katherine,
We also _know_ that the worst alarmist worries about CD life are
untrue. Besides the 10 year estimate you are quoting, there was someone
here last year with a 3-5 year guess.
Philips and Sony have 12-15 year old disks in their archives which
are still good. Therefore we can guarantee that the critters will last
longer than 10 years anyway<g>.
Frank
Frank,
Agreed, but I didn't want to sound too hyperbolic. For that matter, there
are one-off CDs around that are 8-9 years old (Taiyo Yuden's early
efforts).
Kathy <<– The One-Off CD Shop MidSouth, Inc.
Kathy,
Your approach requires a one-off CD-ROM maker. They are still
expensive although they are coming down in price. Also, I belive that the
one-off CD-ROMs are made through a photo-chemical process. Wouldn't they
be less stable than the so-called glass master discs?
Phil
Philip,
So far none of the one-off discs has died of old age. That's not to say
they won't ever do so, of course. It's pretty hard to predict until some
have gone bad. The dye is quite stable, and as long as you take reasonable
care (don't leave it in the sunlight, don't expose it to writing-wavelength
lasers again, don't cause cracks in the plastic, etc. which are the same
precautions you should take with any CD), they should last indefinitely.
The best made CDs are well sealed against air, moisture, etc. so there
shouldn't be any oxidizing going on inside the disk as long as the seal is
intact. The same is true of manufactured CDs. If the lacquer is scratched
or if the polycarbonate is cracked, their metalized layer can oxidize and
become unreliable, too.
BTW, the glass master is something used during the manufacturing process —
that's not what you get when you buy a CD or CD-ROM.
<<Your approach requires a one-off CD-ROM maker. They are still expensive
although they are coming down in price.>>
The easy way to avoid the capital investment is to get a service bureau to
make your one-offs (ahem! bias alert!) If you plan to make enough discs
to justify purchasing a one-off system instead of using a service, the cost
isn't going to be a big factor, anyway.
Kathy <<– The One-Off CD Shop MidSouth, Inc.