#Greg's new toy
34 messages in this thread
Thanks Mr. X-Sony dealer.
Paul
P.S. – That new little toy you sent me saved me about 3 hours last night.
Paul:
>> That new little toy you sent me saved me about 3 hours last night.
Great! You were the final test. Well, we might as well make the announcement
here, now that it is official:
Pyros/Grubba in conjunction with Abekas has developed a BXP IPAS routine which
reads and writes out fully color corrected YUV files directly out of 3D
Studio!!! These may be copied directly to an Abekas unit, or may be dumped to
an Exabyte tape drive with a 'PC-tar' program.
A side benefit of these files is that D-1 digital is 16 bits deep, thus making
the file size smaller, in addition to being perfectly color corrected!
There is a working demo of the IPAS loaded into the libraries, the only
limitation is that it only reads and writes 1/2 size files. (I guess if you do
a lot of D-1 tests, you may even find it useful as-is! <g>)
The IPAS routine will be sold for $199.00.
Leave private e-mail if interested.
Greg Pyros
As a testament to Gus and Greg's new IPAS. Anyone who is going out of 3DS to a
DDR needs this software. It works absolutely fantastic and the sheer fact that
it reads and writes yuv files is worth the price tag. It was so nice rendering
directly out to yuv files and having 3DS display them at the same time through
the ATVista. No need to convert files after the fact.
We even pulled up yuv files from the SGI machines directly in 3DS and the
vibrant drivers compute the colors and display them in 256, 32K, or 24 bit
files on your SVGA monitor. For those of you who aren't familiar with yuv
files, it's virtually impossible to view them in any program including
everything on the SGI platform, unless you're pumping them through an Abekas or
Accom. With this new IPAS, just enter the file name and "TADA" it's up on your
monitor.
Gus & Greg ……. thank you thank you thank you.
Paul
Paul:
You're welcome, you're welcome, you're welcome! <g>
Reading and writing YUV files should save lots of people lots of time and disk
space! Quality output is getting easier and easier!
Greg Pyros
Out of curiosity, what is the "PC – TAR" program needed for? What would
be the process of rendering to YUV with your IPAS and putting them to DAT tape
for dumping to an Abekas (provided I can find a sevice bureau that can use DAT
tape… is this a possibility, or is Exabyte the ONLY way to go?)
Thanks,
David
PS: Are you still handling sales of your IPAS routines directly?
David:
Hello again!
The PC-TAR program is only needed if you are going to write directly to an
Abekas through an Exabyte tape drive, which seems like the most common method
of getting these files over there.
Of course, as you suggested, if your service bureau accepts another tape
format, you don't need anything except the BXP IPAS itself!
Greg Pyros
Thanks! I'll have to start checking around to see what my options are.
David
PCTAR is a freeware program (available in the UNIX forum – go UNIX) that will
archive files. TAR is the unix Tape ARchive file format, and since Abekas,
SGI, etc… are UNIX boxes, this is what they can read.
The process would involve archiving the YUV files to tape with PCTAR and
aspitape, then dropping the tape off at your service bureau where the files
could be extracted on a unix workstation, or Abekas. The disadvantage to DAT
is that DAT tape drives are not directly supported by the Abekas DDR's, so the
operator would have to run the files through a UNIX workstation with a DAT
drive attached, charging you more because of the extra work. With Exabyte, the
drive is hooked up directly to the DDR and all you have to do is download
directly from the tape to the Abekas. Also, not all DATs are created equal –
some drives can't read others' output. What you should do is find out how much
the bureau will charge for a DAT transfer (most places I've checked with CAN do
this), and if it's reasonable then you should do a test tape, to make sure that
everything is cozy between your and their drives.
John Tissavary (LUNA cie, inc)
Thanks for clarifying things a bit more. I'll have to call around to
see who will accept DAT backups. I have a PAR, and while it's a wonderful
thing, I would like to have some of my final animations put out via abekas to
betacamSP, just to have them on hand (when I can afford it… IF its
affordable). Or I might just rent a betacam deck for a day and dump right from
the PAR.
I recently was forced to get a high capacity tape drive and despite
wanting an Exabyte, was forced to get a sony DAT because of the price ($2200 or
so for the Exabyte, $900 for the Sony). I do love it though… it will
read/write DDS or DDS2 (120m) tapes and store up to 16 gigs on one tape. Its
fast too, it'll backup one gig in just over 30 minutes (quite a jump from the
Colorado 250, there was no way I was going to try to dump 3.5 gigs of disc onto
QIC80 tapes!).
David
The Abekas is nothing more than a Unix workstation with lots of disk space and
a heck of a transfer rate. As far as it is concerned, anything calling itself
"SCSI Tape" will work. In theory (I know nothing about the Sony DAT unit and if
it is anything like their CDROM, you're out of luck as it is incompatible with
the rest of the world) all you've got to do is to unhook their Exabyte and hook
your own unit. Reboot the thing and off you go.
If the Abekas is networked with something else, and this something else has a
DAT unit, it is pretty straight forward to read the tape and send the stuff
directly over to the Abekas. You can pipe the TAR output straight out to the
Abekas and the data never makes to the local disk.
>> if it is anything like their CDROM, you're out of luck as it is incompatible
with the rest of the world…
I used to have a sony CD drive… the one that claimed to be SCSI, but
would only work with THEIR interface card. The DAT that I have (sony SDT-5000
external, or something like that) is hooked up to my regular Adaptec card, so I
would assume that as long as I disable the hardware compression I should be OK.
If the place is local I can even bring the tape drive to them, and not worry
about anything. Well, I'm not ready to do this yet, so I'll call around when I
get close.
Thanks,
David
Hi Gus, I was just catching up on this thread and wondered if the same would
apply to a Pinnacle drive too?
Michael B
I think you'll find that unless you're planning on dumping more than 2000
frames in a day, and don't mind the lesser quality of the PAR, the Abekas will
be a great solution. BetaSP rentals (PVW2800) run @ $300 per day here in SF,
and you need insurance or a big $$$ deposit to get the deck at all. Abekas
rates should be (Exabyte – DAT may be more) @ $.15 per frame or $200-$250 per
hour. There are more expensive places, like the Tape House (rip off!!!) in NYC
that charges $450 per hour, but you should be able to find a good deal in L.A.
John Tissavary (LUNA cie, inc)
Thanks. It'll be a little while before I need to do that anyway, so
I'll just call around to see who can do what when the time comes. I would be
doing over 3000 frames, so the betacam rental idea might not be a bad one,
considering it might be a bit cheaper that way, and this would be out of my own
pocket anyway.
And just to clarify the PC-TAR thing a bit, does this archive the YUV
frames directly to a tape (4mm, 8mm, or whatever), or would you run it through
PC-TAR and then use your normal backup software to go to tape?
Thanks,
David
David:
Normally, you would run 'tar', which saves it to tape at the same time in the
proper UNIX format to be read on the Abekas.
Greg Pyros
Thanks.
David
No, you have to take care of the YUV conversion since PC-TAR is strictly a tape
archive program. PC-TAR is what gets the files onto the tape drive in a UNIX
readable format.
John Tissavary (LUNA cie, inc)
Wow!
Congratulations Gus and Greg… This is great news!
– G
Gary:
Thank you! Just our contribution to making life easier for the masses of 3DS
users out there.
One of the things I like about it (after struggling with many gigabytes of
background files (8 minutes worth) on the "Digi Digital" project) is that
bringing in full res files in Targa format from the Abekas, between filmgrain
and the live shooting, the 720×486 "compressed" targas don't compress much at
all! Most were still over a megabyte.
The same resolution YUV files are less than 700k!!!
Greg Pyros
Greg,
Your Abekas IPAS sounds interesting indeed. Here's a few straight questions.
Will the BXP allow you to composite them in VP? Is the YUV file data
interchangable with the ASDG YUV data? Is the YUV data written straight from
the renderer or is it converted after the fact? I believed you also mentioned
its color corrected. Could you elaborate on how this is accomplished? Can it be
disabled? What effect does gamma have on the process?
TIA
John
The BXP will allow you to do anything you do with any other file format. It is
totally transparent to the user. As a BXP, YUV is now just another "native"
file format within 3D Studio.
No, there is no "conversion" involved. The YUV file is saved out straight from
raw 3DS data. Of course, the raw data is converted to CCIR 601 from its native
RGB format.
I have never looked at the ASDG program. I think its advantage is using the
Exabyte for sequential dump (tar files cannot be appended and only the new
Abekas code can read tar files). As to the format, it should be the same as
there is no variation allowed in the Abekas for input/output.
There is a fairly big difference in the dynamic range between straight RGB
color space and YUV. The Chroma filter helps to avoid banding or contouring,
particularly on smooth shaded surfaces. You have to remember that we're moving
from 24 bits per pixel down to 16 bits per pixel.
The input gamma value is irrelevant as far as the save/load process goes. You
should treat it just like any other file format. Whatever you give me goes out
the other end. The Abekas spec calls for a 1.2 value though JohnT has found 1.8
to be a "more pleasant" level.
Gus,
I've got the ASDG program but I think that your BXP has some distinct
advantages in that you can write the files directly (thereby saving time) and
use them in VP without having to convert them. Having the ASDG version is nice
for different reasons and I expect the two will compliment each other well.
>> You have to remember that we're moving from 24 bits per pixel down to 16
bits per pixel. <<
According to my digital handbook CCIR 601 specifies 8 bits of information for
Luminance and 8 bits for both Chrominance axes. 8 bits allows a maximum of 256
levels of information to be represented for each component, (ie 256x256x256) as
a result approximately 16 million unique values can be represented in the CCIR
601 system. (Quantel refers to it as an 8 bit system.)
So the question is, is it not just a difference in how the values are stored
and read (Y,R-Y,B-Y or Y,Cr,Cb) as an encoded binary stream rather than
actually only allowing 16 bits per pixel as we commonly associate with a 16 bit
image? In other words isn't this just a more complex process to achieve the
same result for a 24 bit image? This apparent discrepancy has concerned me in
the past and I would appreciate any further clarification especially as it
relates to the internal YUV processing at render.
That being said, I'm curious to know how the chroma filter is applied.
These issues resolved, I'd be happy to send the requisite amount to Mr. P
provided I have the correct address if its different from previously.
Thanks again,
John
It's not that simple. Yes, there are 8 bits for each luminance, R-Y, and B-Y.
The difference is that each pair of pixels share the chroma so it takes two
pixels to get both R-Y and B-Y. Luminance (Y) is unique for each pixel. The
physical layout is 16 bits pixels but you need 32 bits before you can make
sense out of a single pixel. On top of that, the dynamic range is limited. Not
all bits of a signal are used. There are around 220 possible values for
luminance and 224 for chrominance. The 16 million figure is somewhat pointless
as you would need a monitor with a 4,096×4096 resolution in order to show all
of them. This also assumes that no single pixel is repeated.
Still a YUV image should be much more closely representative of a 24 bit image
than a 16 bit image regardless of whether you can represent the full luminance
or chrominance values possible in direct RGB output.
>> On top of that, the dynamic range is limited. <<
Maybe relative to what the renderer is able to output in RGB terms, which may
have been your point all along, but not relative to the video medium as a
whole, where digital generally exceeds source, which is typically Beta Sp.
Anyway, it wasn't my intent to get into a debate about it. My main concern was
that the RGB to YUV calculations were as faithful as possible in conformance
with CCIR 601. The 16 bit analogy, as we conventionally think of 16 bit RLE is
not a relevant comparison as a YUV file is a matrix scheme with filter
characteristics, which purports to achieve the same result as what we
conventionally think of as a 24 bit image; one with a potential for 16 millions
colors.
Also, remember that all less-than-full-bandwidth video schemes are engineered
to take advantage of the design of our eye-brain system. Humans are much better
equipped to percieve luminance and detail information than colors. Formats like
like u-matic, vhs, hi-8, beta, et. al., all strive to record wide luminance
bandwidth and fill between the outlines with fuzzy color. Still images might
look awful, but in motion everything looks right. CCIR-601 (D-1,YUV,4-2-2)
gives you the highest quality possible balanced against the limitations of
NTSC, which itself is designed to economize bandwidth based on our eye-brain
specs. 24-bit RGB (4-4-4) or RGBA (4-4-4-4) is only useful in the production
processes of rendering, painting, compositing, etc. . None of that extra data
can get through the transmitter or be displayed at home on a TV . . . YET.
Bruce
Bruce,
The only environment that concerns me is the production environment. While I
get your point, the fact is there is a noticeable difference between 24 bit and
16 bit regardless of the final medium. It may not make a difference to some,
and of course they won't be interested in acquiring an IPAS that will support
full CCIR 601 implementation of the YUV file format. But its a difference I and
others I know can descern. I've come to expect a very high level of image
quality and color depth from 3DS. It is my understanding that the CCIR 601
specification establishes the parameters by which a YUV file can very
accurately reproduce that same level of detail and quality, more so than is
true of other video formats. Which is in contradiction to Gus's comment that
the CCIR 601 specification has a very limited dynamic range and poor color
depth. On the contrary, the specification allows for a generally higher level
of dynamic range and color depth than most source material, according to
Quantel. Whether you can see all sixteen millions colors on a monitor is a moot
point, as is how the final medium will "look" on trasmission. To me this is
not a cogent argument for limiting or restricting the potential of the image
quality and color depth as it is apparent the specification does not have these
limitations. And I get more than a little suspicious when arguments like these
are used to defend the implementation and approach in the design of a YUV file
format.
John
>> The 16 million figure is somewhat pointless as you would need a monitor
with a 4,096×4096 resolution in order to show all of them. <<
I appreciate your thorough description, Gus, but I find the above bit of
reasoning illogical. The necessity for 16M colors has nothing to do with
showing them all at once. Rather, it has to do with how the colors interact.
For example, fill a 640×480 screen with a pure red graduation of color. Since
red is a primary color, the combinations available for the remaining two
primaries are irrelavant. If there are not enough references to the
intensities of red, banding will be apparent.
Hence, without enough references for individual values of each primary color,
banding will be noticeable (excepting dithering). Add any other primary as a
constant accross the entire gradation and still, without enough references to
the primary forming the graduation, banding will still be evident. For most of
us, 256 graduations for a given hue or primary are about all we can see. It
just so happens that having 256 colors for each primary produces a result of
16M colors — whether we need them all at once or not.
Being mathematically oriented as I am, I argued this point with an art
instructor over the course of a semester. He maintained that 65K colors are
sufficient for photographic quality because you could never fit 16M colors on a
screen. I asked "Why, then, do 24 bit images look so much cleaner?" and
offered my above reasoning as the answer. Neither of us ever yielded to the
other's point of view. Oh, well.
Anyway, thanks a 'million' for efforts on this BXP. It is a usefull tool and
that is what really counts!
— James — MAP —
James:
I hear you – it seems like there would be fewer colors, however, because of the
encoding scheme, D-1 looks pretty darn good!
The Abekas uses the same encoding used on the PAR, MAX, Bandit, etc., the only
difference is that the Abekas is uncompressed, while the others us a modified
JPEG compression in addition to the YUV conversion.
Greg Pyros
>> D-1 looks pretty darn good! <<
I only wish I had more oportunites to view the various formats and outputs so I
could have a better eye towards image quality!
— James — MAP —
The mistake is in corelating RGB directly. Yes, in RGB, if all you try to use
is the red component, you only have 256 possibilities. The YUV color space
isn't divided like that. The chrominance is coupled with luminance as an "added
feature". I guess this is the closest representation of analog data using
limited digital space. When I think of 2^24 possibilities, I'm not dividing the
spectrum in 3 distinct groups of R(2^8), G(2^8), or B(2^8). I'm just raising
the dynamic range of luminance and chroma. Your art instructor would be right
if the 65k colors were linear and not divided into 5 bits each of R, G, and B.
Don't think digital, think analog.
I wrote a program that shows this quite well. Even if you don't ever need it,
it's interesting just to look at the representation of a different color space.
It's a chroma key program where I try to get out of RGB and make you think
exclusively in LHS (Luminance, Hue, and Saturation). Even though I'm dealing
with a 48 bits RGB color space, I use only 8 to do the display. What I do to
get a final "alpha" is to allow you to double mask. This is a trick to allow 16
bits alpha using only 8.
Great explanation Gus !
BILL
I must not have made myself clear about what I was arguing. I was _not_
attempting to argue that YUV is substandard or better than RGB. I _was_
talking about the digital RGB format and the necessity for so many colors.
I have little or no understanding of the YUV format. I was only saying that,
for an RGB display, the fact that N colors won't fit has nothing to do with why
that many colors are available. I have heard the "won't fit" premis many
times, as it relates to digital RGB graphics cards, and it continually strikes
me as a paramount of illogic.
In fact, I would hypothesize that my point is true regardless of the format
(RGB, YUV, CMYK, HLS) so long as it is digitally encoded. To speak
inductively, I would say that a digital encoding system must have enough
references within each component of the digital code to eliminate distinct
separations between areas of pixels a with single differing component.
Further, the fact that the total combination of necessary references per
component exceeds the number of pixels (ie total # of addressable colors "won't
fit") in the display device at once is irrelevant.
For analog, however, the point is nullified since the distance between
references in any given component become infinately small, and the whole idea
of references is invalid.
Anyway, the last thing I want to do is get into an arguement with you. <BG> I
just wanted to point out what I feel is a popular misconception.
— James — MAP —
JE:
>> provided I have the correct address if its different from previously.
We haven't moved! Same Bat-time, same Bat-channel, same Bat-place!
Greg Pyros
Greg, Congrats once again to you and Gus for making new tools available. I am
interested for sure and I also want to know how to get a copy of NDUMP to use
with the PAR. Thanks
Michael B.