Page Updated: Tuesday, October 10th, 2000
MAJOR NEWS!! Senator John Edwards
Introduces
'Spyware Control Act'
What do the NetZip-descended
file downloaders whisper when
they think you are not listening?
OVERVIEW: How Does This Affect YOU ?
As you will see on the page below, if you use the RealNetworks RealDownload, Netscape/AOL Smart Download, or NetZip Download Demon utilities in their default configuration . . .
EVERY TIME you use one of these utilities to download ANY FILE from ANYWHERE on the Internet, the complete "URL address" of the file, along with a UNIQUE ID TAG that has been assigned to YOUR machine, and — in the case of Netscape's SmartDownload only — YOUR computer's individual Internet IP address, is immediately transmitted to the program's publisher.
This allows a database of your entire, personal, file download history to be assembled and uniquely associated with your individual computer . . . for whatever purpose the program's publishers may have today, or tomorrow.
VERY IMPORTANT: When I re-examined my findings in the face of RealNetworks' insistence that I was absolutely wrong about my conclusions, I caught something that I had missed before: My exact personal name and private eMail address was being sent back to RealNetworks whenever I downloaded a file. When I confronted RealNetworks with this, they explained that it was due to the fact that I had purchased a product from them in the past, and the "cookie" my system had received during the purchase was being returned to them.
That certainly makes file downloads seem far less "anonymous" than RealNetworks continues to allege. (Full details are provided below.) | |
|
|
The Saga Unfolds . . .
Friday,
July 14 | I download fresh copies of all three Download Demon-descended file downloading utilities and conduct a series of tests to verify the rumors I've heard about their "phoning home" behavior.
In each case, the behavior I examined resulted from each program's "default configuration" which is enabled unless deliberately disabled by the user. I confirmed that all three programs send a report back to their publishers whenever the program is used to download any file through the Internet. This report includes the full URL of the file being downloaded and an "ID Tag" which could be used to uniquely identify the downloading computer.
In the case of Netscape's Smart Download, the computer's individual Internet IP address is also sent as a "cookie header" which would tend to defeat IP-masking proxies and anonymizers.
|
Monday,
July 17 | By Certified Mail I receive RealNetworks' threat letter — which I ignore because it's just so much nonsense — and proceed to initiate a very constructive dialog with two representatives of RealNetworks. Their V.P. of Government Affairs and Privacy informs me that I am absolutely, totally, and completely mistaken and insists that I immediately take this page down and retract all of my public statements to everyone who has received them. (I guess he must have read Robert Kimball's letter too.)
I refuse to remove the page based solely upon his forceful representations and assurances. But I worry — in the face of their legal threats — that I might somehow have been completely mistaken. So I quickly post a big red notice at the top of this page to notify its readers that RealNetworks is very sure that I am completely wrong, and that I am immediately working to re-verify all of my findings.
Then a much more serious RealDownload
privacy concern rears its ugly head:
It's Monday afternoon, and everything still comes out just the way it did Friday. (In other words, I was right all along.) However, this time I happen to notice that my actual first and last name, and my own private eMail alias address are also being transmitted to RealNetworks as a result of each file download. So I immediately forward the captured packet to the RealNetworks representatives with whom I'm working and ask them what is going on.
By phone the technical manager with whom I'm speaking asks if I've ever purchased anything from Real? I explain that a few months ago I purchased "Real Producer" in order to produce streaming content for my web site. So she explains that my purchase and interaction with their eCommerce server left a "cookie" on my computer which included my real name and personal eMail address from the purchase transaction.
I see. So now my private information — which was obtained by RealNetworks during a SECURE PURCHASE TRANSACTION with an explicit commitment for security, privacy, and secrecy — is being sent back to Real — months later — "in the clear" with no security, every time I download arbitrary files from the Internet using their utility — along with the full name of the file I downloaded and the unique ID that could be used to identify my computer.
I think that's a "Real" problem. And it would certainly seem to contradict RealNetworks' repeated statements that it is not possible for them to associate my use of RealDownload with any personally identifiable information. If my name and private eMail address aren't "personally identifiable information", what is? Moreover, that personal information could be easily associated with the file download which directly triggered the transmission of that information.
Based upon my understanding of how and why this happens, this is easily reproducible and is apparently going on all the time with RealNetworks customers . . . like right now. If what I've been told by the RealNetworks technical manager is true — and it certainly fits the facts and logic — it appears that anyone who has purchased a RealNetworks product through their eCommerce system receives an insecure, plaintext, cookie containing their actual name and eMail address. I certainly did. And this cookie is then sent back to RealNetworks . . .
. . . even in situations where users of RealNetworks'
products have been repeatedly and even forcefully
assured of their absolute anonymity.
Whoops.
|
|
| On a Technical Point:
RealNetworks has stated repeatedly that they care about their user's privacy. And they tell us that they are "the leader in the delivery of Internet media." Monday they told me that they employ 400 programmers. With all that, wouldn't you be inclined to presume that they had a grasp on Internet Technology?
If they care about our privacy, why are they storing
my real name and private eMail address — from an
eCommerce transaction — as "plain text" in a cookie,
and sending it out without any security whatsoever?
Even if it weren't being sent back due to a file download it would still be a significant privacy concern. Why not, instead, use a cookie the way it was intended to be used? A cookie should be an "opaque token"; an apparently meaningless string of characters, which only has meaning to the entity which created it. |
|
But none of that was the problem I was facing at the moment. (Perhaps we'll deal with that one next.) I was working to demonstrate to the RealNetworks representatives the absolute truth of what I'd been saying about the transmission of a system-unique ID.
So, using RealDownload, I downloaded three different files over the course of several hours and from different Internet servers. I captured each resulting 'downloadid' as it was leaving my computer on its way to RealNetworks:
downloadid=9B1450495BF211D4A025002018252799
*
downloadid=9B14504A5BF211D4A025002018252799
*
downloadid=9B14504B5BF211D4A025002018252799
As you can see, they differ by a single character, and that character is changing from "9" to "A" to "B" which indicates standard hexadecimal counting. So I sent these 'downloadids' to the RealNetwork representatives. This apparently puzzled Real's technical manager who said that she'd have to get back to me on it. When she called back she explained that, sure enough, they had succeeded in duplicating the same behavior in their labs and . . . that it must be a bug.
A "bug"?? Yeah . . . okay . . . I guess that would be a big one?
She explained that she had just learned that the last 24 characters of the "downloadid"'s 32-characters, were derived from a Windows GUID.
"GUID" stands for "Globally Unique IDentifier" and is a technology standard specified by the Open Software Foundation (OSF) to create unique and non-repeating "ID Tags". Such "ID Tags" are generated once then stored, typically in the Windows Registry.
If you're really curious, use the Windows "RegEdit" program to look under this key name: HKEY_CLASSES_ROOT\CLSID and you'll see a billion GUID's (Don't change anything!)
In the past, the use of GUID's has aroused the wrath and concern of privacy advocates the world over, since they are like "serial numbers" which can be used to uniquely identify software users.
|
Okay. So now we know how and where RealNetworks gets the last 24-characters of their 'downloadid'. It is a non-changing unique identifier, different for every computer. Today, they may not like the fact that their use of a deliberately unique and fixed identifier has severe privacy overtones, nor that they have been caught in an outright lie about their use of an identifier which is being transmitted and could be used to track the software download habits of their RealDownload users. But I never expected that forcing them to publicly confess the truth would make them particularly happy.
downloadid=9B145049 / 5BF211D4A025002018252799
It appears to be quite likely that the first eight characters are a hexadecimal representation of a 32-bit binary quantity that is incremented for every download — that, in any event, is the behavior I witnessed. So the first portion which appears to be incremented for each download functions like a "download session ID". Whereas the last 24 characters are exactly what I have always asserted: A "download machine ID." Together, they create a deliberately concocted, unique identifier, which, when transmitted from any user's computer could be used to track their users' download behavior over time and to assemble a download profiling database.
|
Tuesday,
July 18 | Things were much quieter today. I was told that RealNetworks staff was "in meetings" most of the day.
Then, at the end of this long day of "meetings" — which were apparently spent carefully wording the following document — RealNetworks produced this formal statement:
|
REALNETWORKS PRIVACY STATEMENT 7/18/00
In response to recent questions regarding certain technical functions of its RealDownload product, RealNetworks today issued the following statement:
"We emphatically disagree with the implications raised by certain members of the technical community about the behavior or planned behavior of RealDownload. To be clear: RealDownload does not transmit personally identifiable user information to RealNetworks without informed consent. It does not monitor users? behavior and it does not log download URL information. Because we do not log download URL information and the product does not transmit registration information identifying the RealDownload user, we cannot and do not store download URLs with personal information — and we never have.
"We work very hard to ensure that our products comply with all of our privacy policies. We have even taken the extra step of hiring Arthur Andersen to independently review our compliance with our own strict privacy policies. Through its eSure audit program, Arthur Andersen has independently verified that RealNetworks does not store URLs transmitted from the RealDownload product.
"Because of the way RealDownload interoperates with the APIs of certain versions of the Windows operating system, it creates for each download a new, 32-character code that does not contain any personal information, but apparently does not fully randomize during each download. Now that we are aware of this technical issue, and because the 32-character code serves no purpose, we are removing it from forthcoming versions of RealDownload.
"As the leader in the delivery of Internet media, we at RealNetworks set for ourselves and will adhere to the highest privacy standards. We appreciate the ongoing diligence of privacy experts and we will continue to develop RealNetworks products in a manner that respects customers? privacy."
|
|
Tuesday Evening . . .
|
Wednesday,
July 19 |
Regarding RealNetworks' Statement:
Since I am in the hot seat here, being the "certain members of the technical community" who has "raised implications", the world will be looking for my reaction to this statement from RealNetworks. I received their statement first from RealNetworks directly, then subsequently from several members of the media. Everyone has wanted my reaction. Here it is:
|
I am unconcerned and unimpressed with most of RealNetworks' Statement. They specifically failed to address the reason for the presence of the "insufficiently random" 32-character code whose very existence they had previously denied emphatically. I am, however, pleased to learn that they have decided that it now "serves no purpose" and will forthwith be removed from the product. The sooner the better for everyone involved. |
|
We are still left with what is, arguably, a much bigger problem: The undeniable transmission of personal and private "personally identifiable" information as a direct consequence of the use of RealDownload. See the full technical 'dissection' below . . .
|
Thursday,
July 20 | Everything I hear from RealNetworks indicates that they are taking every issue I have raised on this page very seriously . . . and not just paying them lip-service, but really doing something quickly:
We'll see what tomorrow brings. Things are looking up. 
|
Friday,
July 21 |
My determination to dig out the WHOLE truth takes an unexpected turn today. Curious about the fact that the size of a full Windows GUID is exactly the same as the size of RealNetworks' infamous 'downloadid', I write my own little program to request GUIDs from the Windows operating environment. Running this program three times on the same computer which performed Monday's results, generates the following three GUIDs:
Three Successive Windows GUIDs WITHOUT reboots
GUID = CCDE2D405EF811D4A025002018252799
GUID = CCDE2D415EF811D4A025002018252799
GUID = CCDE2D425EF811D4A025002018252799
|
|
|
Notice that, EXACTLY like the three successive downloadids generated by RealDownload on Monday, these GUIDs differ from each other in exactly one character, that this character is counting, and most significantly, the LAST 20 CHARACTERS of the GUIDs I generated exactly match the tail of the 'downloadid':
GUID = CCDE2D405EF8 11D4A025002018252799
downloadid = 9B1450495BF2 11D4A025002018252799
|
Next, I use my GUID-maker program to generate three GUIDs, but I restart Windows each time:
Three Windows GUIDs WITH REBOOTS
GUID = A7F1BFC05FD811D4A025002018252799
GUID = 39CC01805FD911D4A025002018252799
GUID = 8ADA6EE05FD911D4A025002018252799
|
|
|
We see that the first 12 characters of the GUIDs are different (especially the first eight), whereas the 20 character GUID tail is absolutely constant, even across reboots of a single system.
Network adapters are designed to possess "globally unique" MAC addresses in order to prevent physical address collisions when communicating across a local network segment. This means that Network adapter MAC addresses are a good source for some guaranteed-to-be-unique "bits". Therefore, the Open Software Foundation's (OSF) GUID creation scheme incorporates the machine's LAN adapter MAC address, when available, into the GUIDs creation. Since the tests have so far been conducted on a networked machine with a LAN adapter, the next logical step would be to perform them on a machine without a network card:
Three Windows GUIDs WITH REBOOTS
and NO LAN Adapter MAC Address
GUID = 7A9196805FE811D4BA1DA6C968FAE763
GUID = 147026E05FE911D4BA1D8FF112DACE63
GUID = 9C1C35205FE911D4BA1DA55166FEC463
|
|
|
As you can see above, without a LAN adapter's static MAC address available, the situation again changes. Now a region in the center of the the GUIDs is static across GUID generation and across reboots, but the last 12 characters, which had previously never changed, are now very different after each reboot.
So What Does it All Mean?
It means this is a big mess. All of the evidence indicates that RealNetworks' 'downloadid' actually is nothing more or less than a standard Windows GUID.
downloadid == GUID
The RealNetworks technical manager told me, Monday, that the last 24 characters of their 'downloadid' were "derived from" a Windows GUID. And while I suppose that's technically correct, it's a bit misleading, since I am now virtually certain that their 'downloadid' is exactly and without 'derivation' a Windows GUID.
"Huh? They're using dynamically generated
Windows GUIDs as their download IDs?"
Yeah . . . I know . . . It is a really weird and dumb thing to do:
As we have clearly seen, it is not reliably static enough to use as a trustworthy per-computer identifier, yet it is one, sort of, most of the time, maybe. But neither is it random enough to be used as an opaque per-transaction identifier (as I believe it was intended) without the serious privacy concerns that I originally raised.
Here's exactly what I believe happened:
The copy of NetZip's Download Demon I analyzed exhibits precisely the same behavior as RealNetworks' RealDownload. Therefore, I believe that prior to RealNetworks' acquisition of Download Demon from NetZip, some programmer at NetZip wasn't the least bit concerned about privacy issues. (This is certainly still more the rule than the exception today.) So this programmer innocently uses a Windows GUID as a convenient unique tag for their Demon's transaction tracking. This programmer never stops to consider, if he or she even knew, that the GUID contains — by design and specification — the machine's absolutely unique LAN adapter MAC address, or some other relatively invariant machine-specific tagging information if the system has no LAN card.
Next, RealNetworks apparently commits two blunders:
 | They employ Arthur Andersen to provide a third-party blessing of a second-party product. Since I doubt that the folks from Arthur Andersen are grossly incompetent, it can only be that they don't really care about, or understand, the nature and requirements for personal privacy. They put the Arthur Andersen eSeal of Approval on a product which is not only sending a unique identifier, but managing to transmit its user's unique MAC adapter address across the Internet while intimately associating it with every file download. Yikes! |
| RealNetworks, for its part, either didn't perform its own effective or useful code review on a second-party acquired product, or it, too, is not sufficiently aware of the requirements for personal privacy. Oh sure, RealNetworks has license agreements, privacy policies, and rampaging lawyers galore, but its actual products suffer time and again from significant privacy concerns.
|
RealNetworks has, undeniably, fumbled their acquisition of Download Demon and the release of RealDownload, but . . .
A completely fair reading of the
evidence suggests that RealNetworks
never meant to violate anyone's privacy.
And, significantly, this is absolutely different from the conclusion I would draw from the design of Netscape's superficially similar Smart Download product. As you will see below, Smart Download creates an ID Tag in the registry of any machine it's installed on and transmits that Tag with every file download report.
|
Tuesday,
July 25 | CONFIRMED: The currently downloadable new version of RealDownload omits the infamous downloadid from its "phoning home" per-file download reports. The reports (enabled by default) continue to be sent, but any user-tracking would be much less accurate now, needing to be based upon the user's potentially dynamic IP address. ("Phoning home" is a fundamentally non-private action for any Internet software.)
CONFIRMED: Previous version(s) of RealDownload continue to retrieve images from RealNetworks' eCommerce server domain. RealNetworks customers who received an insecure personal cookie containing their name and address, will have this private and personally identifiable information transmitted as a result of the use of previous version(s) of RealDownload. I was told this privacy breach would be eliminated five days ago . . . yet it continues.
|
Thursday,
August 3 | RealNetworks continues to bend the
truth and fails to take responsibility
for the behavior of their software.
I receive a copy of an eMail from its recipient. It is reproduced here in full so that the excerpt below can be seen in context. This was apparently generated and sent by RealNetworks' Vice President of Government Affairs and Privacy — the person with whom I have been dealing at RealNetworks.
It may be, at least in part, a form letter sent to anyone who questions RealNetworks about the conduct of RealDownload. If that is the case; if this is the message everyone is receiving; I need to address the glaring inaccuracy it promotes since it is the main topic of concern:
"Unfortunately, recent reports have incorrectly
stated that RealNetworks is capable of tracking
or somehow "monitoring" individuals' downloads. |
If the folks at RealNetworks really believe what they are saying, they must be using a very odd definition of the term "monitoring" since we all know that in its default configuration (unless deliberately disabled by the end user) RealDownload transmits a report for every file that any user downloads, which is received and accepted by web servers at RealNetworks'. And furthermore, by their own admission, they do employ this information for customizing the advertisements which their users' see, based upon the type of file downloaded. They also claim to use this information for other purposes when dealing with "partner web sites." As far as I know, the nature of those "other purposes" has never been clearly articulated. But in any event, it is simply not true that RealNetworks is incapable of monitoring individuals' downloads. They clearly are, and they apparently do.
|
To Summarize before we examine the details . . .
In order to confirm or deny the reports alleging that the Real Networks and Netscape/AOL download utilities might be spying on their users by secretly "phoning home" with detailed reports of every file their users download, I used a readily available "packet sniffer" to monitor the data being sent from one of my machines when downloading a handful of my own website's files.
I was able to quickly confirm that the NetZip-descended downloaders used by Real Networks and Netscape/AOL were, indeed, sending detailed reports of every download "back to base" every time they were used to download a file.
These reports contained the complete Internet URL of the file being downloaded and were accompanied by an apparently unique "ID Tag" which was associated with each machine. To confirm this, I experimented with downloads from several different computers. In every case the "apparently unique ID" being sent out never changed on the same computer, and each computer has its own.
Netscape's Smart Download goes one step further by including the computer's IP address in a separate "cookie" header. This is troubling, since "cookie" headers tend to be left alone as they pass through proxies and anonymizers. This would thwart deliberate attempts at keeping the computer's IP address confidential.
When you consider that each user's computer is uniquely identified, and that reports are being sent back for every file downloaded — and accompanied by a unique ID tag (and, in the case of Netscape, the machine's unique IP address) . . .
. . . It is NATURAL to wonder WHY
this information is being transmitted,
and to what end the data is being put!
Dissecting RealDownload's Packet Traffic
After installing RealNetworks' RealDownload utility, I clicked on a web link to download the file "id.exe" from my server at "grc.com". The following TCP/IP data packet was immediately sent out of my computer to one of Real's servers:
|
![[Home]](/mb-home.gif){kind=small}
![[Products]](/mb-products.gif){kind=small}
![[Services]](/mb-services.gif){kind=small}
![[Freeware]](/mb-freeware.gif){kind=small}
![[Research]](/mb-research.gif){kind=small}
![[Other]](/mb-other.gif){kind=small}
