Microsoft's flawed Linux vs. NT shootout

The test lab looked for help online. But did it really want answers?


On April 13, a test lab named Mindcraft released a report commissioned by Microsoft that declared Windows NT to be far superior in performance to Linux. NT, trumpeted the study from Mindcraft, an “independent testing lab,” is “2.5 times
faster than Linux as a file server and 3.7 times faster as a Web server.”

Numbers like that can get Linux lovers hopping mad. But what really has the Linux community incensed is the way Mindcraft apparently went about seeking information and help from Linux experts.

Sure, Mindcraft — either itself or through intermediaries — did ask for help
in an Internet-based discussion forum. But it did so under false pretences:
It didn’t tell its would-be helpers that it needed the information because it was comparing the performance of Windows NT and Linux at the commercial request of Microsoft. And its request for advice originated from a computer with a
Microsoft address. It refused to
answer any requests for more information about its test. Finally, it apparently promised to redo the test with the input of Linus Torvalds himself — but isn’t giving him the info he needs to do the job right.

The original news of Mindcraft’s test sent the Linux world into immediate convulsions — the sound of
open-source geeks gnashing their teeth could be heard all over the Internet. Were all those claims for Linux’s
technical superiority just so much overheated hooey?

Not so fast, said the Net. As demonstrated in a recent href="
by Linux front man Eric Raymond, if there is one thing that the
Internet is really, really good at, it is marshaling the combined forces of
thousands and thousands of watchful eyes.

In combination with some
aggressive trade press reporting, > numerous details undermining the Mindcraft report soon emerged. First, the study had been commissioned by
Microsoft — a fact not mentioned in the original press release, and buried deep within the Mindcraft Web site. Second, the test had been able to take advantage of extensive
“tuning” information from Microsoft — details on how to make NT work
especially well on the particular hardware configuration used by
Mindcraft. And third, Mindcraft’s attempts to obtain similar tuning
information from the Linux community were half-hearted, at best — and here is where the story gets interesting.

The Linux community swears
by its accessibility, by its willingness to help answer questions in a
variety of forums. But it isn’t perfect. The convoluted story of Mindcraft’s
attempt to get information from one of the most popular such forums –
Usenet newsgroups — illustrates a key Linux weakness, the lack of
centralized sources of information on the newest and most advanced technical issues.

But at the same time, the story
underlines the essential worthlessness of commercially sponsored
comparison tests. The purpose of these tests is to please the customer who
commissions them. Why expend too much energy attempting to find information
that your customer probably won’t appreciate?

The trail starts with a post to Usenet on March 11 by a correspondent
known only as “Will.” The post — which was cross-posted to
the newsgroups comp.os.linux.setup, comp.os.linux.networking and
comp.infosystems.www.servers.unix — asked for information on how to maximize
performance from a Linux-Apache system on an extremely high-performance, multiprocessor hardware setup. And it received one lengthy post offering technical advice and asking for more details — which were not forthcoming.

Nearly a month later, a perceptive Usenetter noticed that the hardware of
that setup matched exactly what Mindcraft had been using for its tests.
And, most intriguingly to the conspiracy minded, the
“message header” for the Usenet post from “Will” included information
indicating that the message had been originally sent from a computer on
Microsoft’s internal network.

I sent “Will” an e-mail asking if the situation he had described in
his Usenet post indeed referred to the Mindcraft test configuration. I
also asked him why he had chosen not to identify himself.

The correspondent acknowledged that he was acting as an “intermediary” between Mindcraft and Usenet. As to his reluctance to come clean, he wrote:

“Any form of post comparing NT vs Linux, or by or would have yielded nothing but flaming insults (I’ve seen it
all too often). I’ve personally been running Linux for some time as well as
NT, and regardless of my personal opinion, it is appallingly clear that
Linux fans are deeply attached to their OS on an emotional level akin to Mac
users. So much so that communications often degrade to mindless sarcasm and

“Will” then noted that Mindcraft was in the process of “rerunning these
tests (and others) with the direct aid and assistance (albeit remotely) of
the top minds in the Linux community.”

By consensus, the two “top minds” of the Linux community are Linus Torvalds and Alan Cox. Cox and Torvalds acknowledged that
they had been in communication with Bruce Weiner, the president of
Mindcraft, but both expressed dismay at the extent to which they were being
allowed to participate.

“I’ve traded a couple of e-mails with Mindcraft people about this,” says Alan
Cox. “They seem solely intent on trying to re-create their existing
pro-Microsoft results and hoping, by attaching some kind of ‘Linux top mind’
credibility to it, they can do more damage.”

“The whole thing has been fairly painful,” says Torvalds. “Mostly because
these people don’t actually let us know what the hell they are doing. We’ve
been offering to be on site to see what the hell is going on, but so far
they’ve refused.”

“There isn’t that much we can do — they don’t actually allow anybody access
to the dang thing, so while they have e-mailed us about their problems, we
can only tell them that the results they are seeing are extremely low, and
we can only guess at the reason why,” continues Torvalds. “Experiments on
similar (but not the same) hardware has shown much better performance in
other places.”

“Will” refused to provide his full name or employer, although he denied that he
is a Microsoft employee. He explained the issue of his mail originating
from a Microsoft mail server by saying that he has “friends at Microsoft”
who “owe me favors.” One such favor, he said, was to host his machine –
which, he noted, runs Red Hat Linux.

Getting up-to-date information about Linux
configuration issues for cutting-edge hardware is a serious issue for the
Linux community. But even if Mindcraft had identified itself, right off the
bat, to Usenet, or Red Hat, or Torvalds himself, it’s not entirely clear
that the community could have come up with answers that would have
significantly improved Linux’s performance.

Mindcraft did not return repeated phone calls and e-mail queries,
but at least one source indicates that the company is tiny, perhaps no more
than two or three employees. Mindcraft’s business depends on satisfying
customers like Microsoft who request specific numbers on specific hardware and software combinations. As one engineer familiar with the testing business notes,
“The choice of what to test and which results to publish are always
driven by the customer’s interests … There’s plenty of money in performance
testing, as long as it’s possible to produce credible numbers that are
favorable to the customer.”

Credible or incredible? The numbers certainly are favorable. One wonders,
though, whether the fallout from this episode will be exactly the
opposite of what Microsoft might have wished. The Linux world is suddenly
highly motivated to accomplish two new challenges — centralizing access to
“tuning information” and upgrading Linux performance on the kind of advanced
hardware that hasn’t been Linux’s priority. Stay tuned.

Andrew Leonard

Andrew Leonard is a staff writer at Salon. On Twitter, @koxinga21.

More Related Stories

Featured Slide Shows

  • Share on Twitter
  • Share on Facebook
  • 1 of 14
  • Close
  • Fullscreen
  • Thumbnails

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and Adam, "Pilot"

    One of our first exposures to uncomfortable “Girls” sex comes early, in the pilot episode, when Hannah and Adam “get feisty” (a phrase Hannah hates) on the couch. The pair is about to go at it doggy-style when Adam nearly inserts his penis in “the wrong hole,” and after Hannah corrects him, she awkwardly explains her lack of desire to have anal sex in too many words. “Hey, let’s play the quiet game,” Adam says, thrusting. And so the romance begins.

    13 of "Girls'" most cringeworthy sex scenes

    Marnie and Elijah, "It's About Time"

    In an act of “betrayal” that messes up each of their relationships with Hannah, Marnie and Elijah open Season 2 with some more couch sex, which is almost unbearable to watch. Elijah, who is trying to explore the “hetero side” of his bisexuality, can’t maintain his erection, and the entire affair ends in very uncomfortable silence.

    13 of "Girls'" most cringeworthy sex scenes

    Marnie and Charlie, "Vagina Panic"

    Poor Charlie. While he and Marnie have their fair share of uncomfortable sex over the course of their relationship, one of the saddest moments (aside from Marnie breaking up with him during intercourse) is when Marnie encourages him to penetrate her from behind so she doesn’t have to look at him. “This feels so good,” Charlie says. “We have to go slow.” Poor sucker.

    13 of "Girls'" most cringeworthy sex scenes

    Shoshanna and camp friend Matt, "Hannah's Diary"

    We’d be remiss not to mention Shoshanna’s effort to lose her virginity to an old camp friend, who tells her how “weird” it is that he “loves to eat pussy” moments before she admits she’s never “done it” before. At least it paves the way for the uncomfortable sex we later get to watch her have with Ray?

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and Adam, "Hard Being Easy"

    On the heels of trying (unsuccessfully) to determine the status of her early relationship with Adam, Hannah walks by her future boyfriend’s bedroom to find him masturbating alone, in one of the strangest scenes of the first season. As Adam jerks off and refuses to let Hannah participate beyond telling him how much she likes watching, we see some serious (and odd) character development ... which ends with Hannah taking a hundred-dollar bill from Adam’s wallet, for cab fare and pizza (as well as her services).

    13 of "Girls'" most cringeworthy sex scenes

    Marnie and Booth Jonathan, "Bad Friend"

    Oh, Booth Jonathan -- the little man who “knows how to do things.” After he turns Marnie on enough to make her masturbate in the bathroom at the gallery where she works, Booth finally seals the deal in a mortifying and nearly painful to watch sex scene that tells us pretty much everything we need to know about how much Marnie is willing to fake it.

    13 of "Girls'" most cringeworthy sex scenes

    Tad and Loreen, "The Return"

    The only sex scene in the series not to feature one of the main characters, Hannah’s parents’ showertime anniversary celebration is easily one of the most cringe-worthy moments of the show’s first season. Even Hannah’s mother, Loreen, observes how embarrassing the situation is, which ends with her husband, Tad, slipping out of the shower and falling naked and unconscious on the bathroom floor.

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and the pharmacist, "The Return"

    Tad and Loreen aren’t the only ones to get some during Hannah’s first season trip home to Michigan. The show’s protagonist finds herself in bed with a former high school classmate, who doesn’t exactly enjoy it when Hannah puts one of her fingers near his anus. “I’m tight like a baby, right?” Hannah asks at one point. Time to press pause.

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and Adam, "Role-Play"

    While it’s not quite a full-on, all-out sex scene, Hannah and Adam’s attempt at role play in Season 3 is certainly an intimate encounter to behold (or not). Hannah dons a blond wig and gets a little too into her role, giving a melodramatic performance that ends with a passerby punching Adam in the face. So there’s that.

    13 of "Girls'" most cringeworthy sex scenes

    Shoshanna and Ray, "Together"

    As Shoshanna and Ray near the end of their relationship, we can see their sexual chemistry getting worse and worse. It’s no more evident than when Ray is penetrating a clothed and visibly horrified Shoshanna from behind, who ends the encounter by asking if her partner will just “get out of me.”

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and Frank, "Video Games"

    Hannah, Jessa’s 19-year-old stepbrother, a graveyard and too much chatting. Need we say more about how uncomfortable this sex is to watch?

    13 of "Girls'" most cringeworthy sex scenes

    Marnie and Desi, "Iowa"

    Who gets her butt motorboated? Is this a real thing? Aside from the questionable logistics and reality of Marnie and Desi’s analingus scene, there’s also the awkward moment when Marnie confuses her partner’s declaration of love for licking her butthole with love for her. Oh, Marnie.

    13 of "Girls'" most cringeworthy sex scenes

    Hannah and Adam, "Vagina Panic"

    There is too much in this scene to dissect: fantasies of an 11-year-old girl with a Cabbage Patch lunchbox, excessive references to that little girl as a “slut” and Adam ripping off a condom to ejaculate on Hannah’s chest. No wonder it ends with Hannah saying she almost came.

  • Recent Slide Shows



Comment Preview

Your name will appear as username ( settings | log out )

You may use these HTML tags and attributes: <a href=""> <b> <em> <strong> <i> <blockquote>