Notes or Exchange? Let's put an end to those creative statistics
by Volker Weber
Ed asked yesterday:
I don't see why you are constantly dismissive of the number of customers stat.
The answer is simple. Statistics that seem to contradict each other make me curious. So let's put an end to the discussion how many of the worlds largest corporations use Notes or Exchange.
Go to this wiki: http://dominoorexchange.pbwiki.com/, state your name and mail address and use the invite key "that is the question".
Now you can look at the current list and add your own knowledge. We should be able to complete this list in less than a day.
Comments
You do realize you're comparing apples to oranges, right?
"IBM claims more than half of the Global 100 use Notes and Domino software. Jim Bernado of Microsoft claims that 84 of the Fortune 100, and 71 of the Global 100 use Exchange as their email system. It's time to find out what it is."
IBM claims usage of their software not just for email and Jim is talking exclusively about email, so it means both are right when talking about HBOS for example:
39. HBOS: Exchange (70k Notes application users)
You are asking people to list only the primary mail system but this has nothing to do with what IBM is claiming.
Jim has said: "84 of the Fortune 100, and 71 of the Global 100 use Exchange as their email system. That's not folks thinking about it, and that's not mixed shops. That's companies using Exchange for all of their employees who have email."
To which Ed responded: "As for your claim of 84/71...for "all of their employees of who have email". Pretty bold claim, especially since it's inaccurate."
So we have a dispute over the 84/71 figure. We're just trying to see what the answer is. If we can identify 30 of the Global 100 that have at least some Notes users then we know that Ed's assertion, that Jim is inaccurate, is true.
I'm sure we can figure out the Fortune 100 as well.
@ Vitor - The problem is that some within IBM are using this in discussions with customers who are considering migrating from messaging.
If it were just applications I have no doubt that there's some Lotus Notes apps hidden away in many companies. Sybase can probably claim the same penetration with PowerBuilder ... the fact that the majority are built using PB6, on a box that hasn't been touched in years to support an application used by 2 users is seemingly unimportant... and if Sybase were to build a messaging system with a fat-arse client on Eclipse I wouldn't buy into it simply because they had 50% penetration of PB.
What's most interesting is that in choosing his response - Ed opted once again to personalise this, and disparage the reputation of every other analyst except the one that suited his view.
I applaud Volker's attempt to bring clarity to the situation.
Jade, it's not personal, it's looking at the full picture.
Why is it not relevant that a certain Fortune 100 company runs their business on thousands of Notes applications, building more each day, but happens to run their e-mail system on Exchange?
Why is it acceptable to take a published report at face value, and not ask questions the methodology behind it (e.g. Ferris)?
Why do you think it is that you have Jim saying one thing to support Gartner but nothing about the IDC report (that's the one that shows MS losing share from 2006 to 2007)? Gartner surveyed 400 companies in English on e-rewards.com in Q4 2006 and came up with a market share report That is somehow supposed to be taken at face value, and the fact that MS does so isn't supposed to make you wonder?
@Ed - "Why is it acceptable to take a published report at face value, and not ask questions the methodology behind it (e.g. Ferris)?"
It is acceptable to ask questions, isn't that what Volker's advocating?
@Ed - "Why do you think it is that you have Jim saying one thing to support Gartner but nothing about the IDC report"
No idea, perhaps he doesn't like to make disparaging remarks about other analysts or customer's decisions (see comment #10).
Jade, thanks for reading my blog. I look forward to your contributions there. I realize that not every vendor (or every staff at a vendor) is willing to challenge opinions, but I'm equally willing to be proven wrong.
In this particular exercise, though, I expect the end result will be more favorable to IBM than to Microsoft's assertions.
There are so many more meaningful things that have to go into these kinds of surveys.
At the Fortune 1000 size, many if not most of these companies will have parts of the organization on different platforms, will have mergers and acquisitions in various states of integration, and will have people from different units with different answers.
To get good info, you need to know how many shops are 90% or more using Notes Client to Domino vs. Outlook Client to Exchange. Of those, you need to know how many are using the latest versions of each, and of those who are not how many are planning to go to the latest version within a year.
THEN you can start to compare Notes Custom Apps vs. VS.NET custom apps, Office Communication Server vs Sametime, and Sharepoint vs. Quickplace/Quickr/Connections (I'm still not sure which is the real comparison).
Don't even get me started on SQL Server vs. DB2. F'ing SQL server is required at the workstation level now on more and more Microsoft products. Does it count when its embedded like that? What about counting Connections users as WAS users? Technically true, but accurate?
These questions are meaningless, but sell newspapers and blog hits.
These questions are meaningless, but sell newspapers and blog hits.
Don’t forget that they also sell products, which is why the question is being asked: for as long as vendors bandy numbers around, people will want to get behind the data.
Sample size is everything in a survey. In political polling, the pollster nearly always declares the "margin of error" in their results, which is tied to the sample size: the less people surveyed, the more inaccurate the results are likely to be.
Even more interesting is when they report "crosstabs" (which subdivide the results into demographics of some kind) but don't publish the actual numbers involved. For example, a recent U.S. presidential poll conducted by SurveyUSA found McCain leading Obama 55% - 45% among African-Americans in Iowa; many who saw those results didn't trust them, and with good reason: because the African-American demographic in Iowa represents only about 2% of the total, this particular crosstab result is likely based on a weighting of 8-12 interviews. So even if McCain only has 7 African-American supporters in Iowa - which is improbable, of course, but not impossible - it's possible that in only talking to 12 African-Americans you'd accidentally stumble upon every supporter he has in that demographic, but the sample size would completely skew the results... again, improbable, but not impossible. Hence, every poll has at least some inaccuracy, because the only way to know for sure is to actually hold the election.
Surveys conducted by Ferris and IDC are not exempt from these rules. Low sample size, high margin of error. So Ed's got a point: the more we know about a survey's methodology, the more we can surmise about how inaccurate the results are... and why. But so does Volker: IBM knows exactly who their customers are, and how many seats they have licensed. they've already "held the election", so to speak. They're just not releasing the full results, which encourages speculation about them to continue. I'm sure they've got a bevy of reasons for describing the numbers the way they do and have... I'm just saying both sides have a valid argument here.
The freudian slip in the question is interesting.
The question is "Notes or Exchange?" though it would more properly be "Domino or Exchange?" or even "Notes/Domino or Outlook/Exchange?"
Of course, Exchange was never called Outlook Server while Domino was once - a long time ago - called Notes Server, and the question is corrected on the wiki page itself although all of the answers there are then also given as either Notes or Exchange.
Before anyone accuses me of pedantry, I will defend myself by restating that I think it interesting that the comparison being drawn is between Notes and Exchange, each being by far the weaker of the pair to which they belong.
Subconsciously, people want to defend Notes. It just doesn't occur to them to speak up for Domino. Could this be because that Notes actually does need all the help it can get, where Domino is more than capable of speaking for itself?
Personally I merely tolerate the Notes classic client, warts and all, because it still works well even though it looks a bit old and tired. I actually dislike new (Eclipse) Notes increasingly because it gets in my way and slows me down, and still seems a little flaky (yet more Sametime wrangling this morning, for example, which just shouldn't be necessary), though 8.0.1 is way better than 8.0.
Domino, on the other hand, continues to surprise and delight me. It still impresses me that a single server can do so many different things (web server, directory server, email server, workflow engine, yes, border MTA and so on and so forth) and do them all well, reliably and securely.
Outlook, which I have had occasion to use recently, is actually not a bad email client. That's all it does, so it is unwise to compare it directly with Notes, but it is reliable and easy to use.
And finally, I can think of no feature or attribute of Exchange that isn't done better by many other products. Exchange is merely a me too email server product.
Given all of this, I would actually expect to find a fair number of hybrid environments.
Where most hybrid environments fail is in the area of integration. There generally isn't much, but here's an idea.
What about Domino server, running LDAP, IMAP, SMTP and HTTP, with Outlook clients for mail and workflow apps delivered via a browser?
We get a secure, reliable, scalable server and app dev platform and the users get a familar and trusted UI.
Just a thought.
Keep the flames down to a low simmer if you can. I'm rather sensitive.
It's not a slip. I chose those two words very carefully, because the Domino branding never stuck. People understand when you are asking "Notes or Exchange".
Domino branding as I recall (I was there) was a bit of an accident - it was originally the beta project name for the IBM HTTP stack that was grafted into Notes Server 4.6. It has sort of half stuck, but lets face it IBM's track record with product names is not particulary glorious.
Users see Notes or Outlook, in fact they often tend to think 'Lotus' or 'Microsoft' when you speak to them about email
From these confused roots (both MS and IBM) we move to confused capabilities, the classic Domino v Exchange argument which can be used to prove black is white or night is day.
However some truths are absolute:
o Notes/Domino once led the market, it does not anymore
o In some countries MS have been much more successful than others poaching IBM accounts
o IBM could not market itself out of a paper bag (sorry Ed, nothing personal)
o If we knew the exact number of licences that are installed today for both platforms it would not make the slightest bit of difference to the eventual outcome
We can discuss the details and the words, but Volker's idea seems to work well and we are already seeing interesting results and useful comments.
So thanks for that from my side :)
@Jade: The problem in truth is that both IBM and Microsoft use such numbers to try and assert one is better than the other.
Microsoft generally gets a pass on this kind of behaviour by most analysts & media outlets. I challenge you to not let Microsoft get away with this while you call out IBM for it. Same thing goes for whether someone cares to support one report and not another. It isn't proper to call out Ed for it but give Jim a pass.
Questioning how a report is gathered & prepared is entirely a valid exercise. How many times have reports been paid for by a vendor to show them as being the best option?
Having said that I am completely in favor of clarity, since it seems all we can ever do is talk about market share of one vs the other instead of building solutions that are right for the customer - even if we do have certain biases.
I think this list is coming together just nicely. Thank you all for contributing.
I think its becoming quite clear from the list that many of the very large companies use both Notes and Exchange, and the "primary" email system in use within that organisation will depend on who you ask.
Taking Boeing for an example, there is a case study at the Microsoft web site citing them as an Exchange customer, but I know for a fact that they do have several Domino installations. This kind of cloudy picture in the large multinationals does not make for clear statistics. Sony is another similar example, they use Notes at Sony Pictures/Entertainment, but I believe Exchange is used elsewhere.
I just grabbed the page source, and selected the entire "", popped it in notepad, and saved it as an xls. Opened it in excel and used these two formulas:
=COUNTIF(A1:A176, "N")
=COUNTIF(A1:A176, "E")
As of my writing this, I showed 28 N's and 30 E's for what that's worth.
I question the accuracy, but its an interesting perspective. I've give vowe credit for that.
* Damn html limiting thing. I mean tto say I grabbed the entire table. I used brackets though and it was removed. Didn't see it in the preview because I'm a bonehead like that.
Lets see if this works: <Table> -- ok. YOu have to use & then lt then ; together like escaping html does.
-Don't even get me started on SQL Server vs. DB2. F'ing SQL server is required at the workstation level now on more and more Microsoft products. Does it count when its embedded like that?
isnt most the market based on revenue , sql express is free so shouldnt count against db2 marketshare
It is worth noting that if Jim Bernardo's reference to "84 of the Fortune 100" was to the Fortune US 100, then 41 of the firms on that list are not on the Wikipedia page that was the source for this list. That page lists global firms with revenues greater than $US40B, whereas #59 in the Fortune 100 falls below that threshold. So while this is very interesting and relevant, we would need to use a longer list of companies to fact-check that part of his claim. Someone has already started doing that by adding DuPont to the pbwiki page even though it is not on the Wikipedia page.
@Ian: Domino was the code name for the web server (as Notes was the code name for what became... Notes). It was initially a separate download and worked with Notes 4.1 server and then integrated in 4.5. Lotus Marketing wanted to call it something like "HTTP services for Notes" but since the Domino name was already out there, it stuck. There was a conscious effort to brand the client release as Notes and the entire server release as Domino. But adding confusion to it all is the designer client which was also branded as Domino. But I think it was to make a distinction between the client and everything else.
Richartd, add a page with the Fortune 100 to the Wiki if you like.
Quite depressing reading on that wiki...and its only going to look worse in another year or two.
I start a new job in July..full time Notes dev/admin. Someone please tell me this wasn't a bad career move!
@Volker: One list at a time is enough. Let's let this one get well-populated, then maybe consider extending out to the Global 500 or 1000.
Oh, what the heck... Somebody else has added two more companies to the list. I'm creating an "Additional Companies" page as a scratchpad for people to do that; and then later if I'm sufficiently motivated maybe I'll try and organize it into a Global 500 page.
@Colin It is an excellent move. Congratulations on your new job.
@vowe
This is fun, thanks for taking this initiative. This puts a good perspective on the marketshare discussion and I agree with Richard to keep this alive and maybe extend it. I will add / clarify where I can based on public reference materials (casestudies / press)
I am active in Lotus Notes for over 15 years. I have never understood, and still don't understand, the choice for branding "Domino" next to "Lotus Notes":
1) is it two different products?
2) does one need the other in order to operate properly?
3) does it perform different functions?
4) does the user percieve it as different products?
5) does the technical press understands the difference?
Why is it necessary to even have to discuss this among experts? I would be in favor of dropping the "Domino" name all together. No more Domino Web Access, but just something like Lotus Notes Webmail. No more Domino Server, but Lotus Notes Mail Server or Lotus Notes Database Webserver.
Branding to me is communicating on what a product does for you, getting this into the heads of the people and having them a positive feeling about it. If you confuse people with unclear names, you don't succeed on the first. If you use two names for the same product, it is harder to get it in the heads of people. If people don't understand what a product does, it doesn't stick in their heads and they won't be able to get a positive feeling about it.
It has been said before.
@Felix Maybe so, but to change the branding now after so many years wouldn't help either...
Sorry people I should not have brought up that branding 'can of worms' here. Let focus on the "we's" and "not we's" instead.
I think the survey is very interesting and might be of use to many parties.
1. IBM will have a view that is not from their CRM / Passport system - it would be nice to know how they differ
2. Microsoft will have a fairly good list of who to target - they are not as efficient as you may think
3. Google will be looking at the market and salivating, and the market will be looking back thinking where are you and what have you really got to offer?
4. Apple might think to worthwhile to give IBM some love (you never know)
5. Ed will have some public figures to beat Jim over the head with (and vice versa)
6. Volker will get more hits to his site (because he's worth it)
7. Life will go on...
As one of my favourite TV shows would say "This (Exchange dominance) myth is BUSTED".
I like that the power of participation on the web has enabled this list to be compiled faster than any analyst firm could. However, I'd like to see the wiki page by readable without ID/passcode. I understand it for editing, but I don't like to log in just to read it.
I think the list is a powerful statement about the power of collaboration. I am also very happy about the links put in there to public references backing up assertions.
What I am waiting for to show up is some of those Exchange to Notes migrations we hear about.
That seems like "moving the goalposts". You never asked for documentation links at the start of the exercise. I'm sure they can be provided to add to both sides of the story.
At any rate, your exercise was to prove or disprove Microsoft, and as Alan says, the wiki results speak for themselves...Microsoft's assertions have been clearly disproven. The effort has indeed "put an end to those creative statistics"... I would have expected you to take credit for doing so.
Lotus has been winning migrations from Exchange or Groupwise or other systems, but they have admittedly not been in the Fortune 100 segment of the market. But we've had a 200,000 seat net-new win in 2008, and several other big ones (including a consolidation that was not driven by a merger/acquisition scenario). Further, we have had very big wins where existing customers have evaluated their current Notes investments and decided to continue forward, as well as adding other Lotus/IBM technologies to the mix.
I think we have achieved one goal very well, and that is bringing some transparency into the market of very large enterprises. Which is one segment that IBM always has been very strong in. Providing equal insight into SMB would be impossible in this manner.
For migrations we could put up a separate page in the Wiki. Those would not necessarily be bound to the Fortune 100 segment of the market.
Currently there is not much participation by the Microsoft ecosystem, but that could change of course.
Currently there is not much participation by the Microsoft ecosystem, but that could change of course.I saw plenty of Microsofties in the list of those accessing the wiki yesterday (De Haas, Cornely, for example).
@Volker - If you put up a separate page in the Wiki for migrations, please consider columns for migration goals (messaging, apps, or both) and the actual results of the migration.
I've heard too many horror stories of companies intending a complete messaging/apps migration form Notes/Domino to Exchange/Sharepoint/.Net who are now stuck running both platforms several years later.
@Volker I wouldn't say we have achieved one goal very well. There are companies listed twice and there are links to case studies that cover 2000 users in companies with over 80000. Can't see this as bringing any transparency. I'm out.
Time Warner is listed as a Notes shop but I know with absolute certainty that CNN is an Exchange shop, as are all the Turner acquisitions. So at the very least it should be listed as mixed, unless Time Warner, Inc. is differentiated from Time Warner and all its operating divisions.
Vitor, no need to take your toys and go home. This is a wiki. Its accuracy is improved by the users.
Thank you, Charles. Use your powers to correct other people's mistakes. :-)
Volker, I ran the Notes list through Wordle to create a tag cloud, printed it up and put it front and center. Oddly, yesterday I had a conversation with an Oracle programmer who was giving me some grief about Notes. It was fun to post. Thanks !
I'll ask again, does pbwiki require that you have to enter a password just to read? You've certainly given IBM a hard time over the years about using ID/PW to access things on IBM.com! ;-) I'd love to point more people to this as a reference if it were directly accessible via a URL.
I disagree with Vitor, as I believe that this exercise gives fantastic transparency, and it should prove reasonably easy to estimate an approximate number of end users as we have access to reasonably accurate staff counts for these organisations.
We can also begin to estimate the number of users operating in organisations with mixed platforms for Notes and Exchange, even if the percentage mixes are only approximate. There is always going to be issues with arriving at exact numbers, but this method is at least as objective as the best methodologies used by consultancies that do this analysis for a living as it's based on numbers that can be verified and tracked.
So far it seems to be verifying that the IDC Report was right on the money, but it also provides an interesting insight into the real struggle that Microsoft is having to extract legacy Notes applications from seemingly committed Exchange shops. Many of the Exchange shops in this list started the transition from Notes over 5 years ago.
Perhaps if more organisations considering a switch from Notes to Exchange understood the actual time and effort involved to complete such a migration, they would act more rationally, or at least challenge the truth of some of the Microsoft sales pitches. In truth, many Lotus Notes applications seem to stick to large enterprises like shit to a blanket.
Is this resistance to migration simply the result of zealous overselling by the Microsoft staff or "independent" consultants, poor migration tools, a lack of skilled migration expertise, or is it an indication that there is still a significant functional gap between the Lotus and the Microsoft collaborative platforms?
The other thing that I like about it, is that because this list is being actively maintained and challenged by both Lotus and Microsoft supporters, it fosters more objectivity and therefore more credibility, a credibility that will only grow over time.
Personally, I would love for this to extend to the top 50,000 organisations around the globe, so keep on updating this list.
Alan,
Try this LINK:
http://dominoorexchange.pbwiki.com/Fortune+Global+500+%282007%29
Thanks, Ian.
This data collection will be more powerful, if we try to leave out interpretation attempts. it's not far from "shit on blanket" to "asbestos". Transparency is good.
Alan, you are being ignored on purpose.
The invite key exists so that people sign up to the wiki. People who have signed up with a working mail address automatically get updates to the wiki. This ensures that a community grows which guards the data against manipulation by less well-minded individuals. The larger this community is the more accurate the data will be. People who do not want to be notified can turn off notifications easily, but it's on by default.
The wiki is less than two days old. Maybe you can bear with us for a little.
Meanwhile we are hard at work building a more rigid access control which requires you to fill out arbitrary information about your company and establish a genuine id and password, that you forget all the time. It will serve no other purpose than to annoy the hell out of you.
What have Wordl.net and this wiki in common?
They can't be reached from Beijing. Dunno if it is the great firewall of China or the sites blocking Chinese IP. To bad I'm here.
:-( stw
@Jade: Regarding your comment- "If it were just applications I have no doubt that there's some Lotus Notes apps hidden away in many companies."....
I sell a suite of products that analyze messaging (Notes and Exchange) and applications (Domino, Sametime, Quicker, Quikr) and I can tell you with great confidence that you *greatly* underestimate the use of Domino applications at some of these companies.
Most of my Fortune 500 customers who have Domino applications have thousands (some have tens of thousands), not one or two.
Generally, if a company uses Exchange for email and Domino as an application platform, it's because the applications are mission-critical, there's no financial business case to migrate and not a good platform that provides the same capability.
If we're going to talk about manipulating the truth to suit our purpose, why is it that we're to ignore the fact that Domino is a good/great application platform that large enterprises continue to use, while Exchange doesn't provide this same capability? Why must the playing feel be shortened to include only email?
I congratulate Microsoft for the well publicized conversion wins of the last few years, but my gut says that in the end, the Fortune 500 list will show that it's still pretty much 50-50 (which is great! competition keeps innovation alive!), even if you throw out companies that use Exchange AND Domino.
Kudos to Volker for helping us to settle this matter once and for all. Why hadn't we thought to do this before?
Because the idea is too simple. :-)
this is a late comment to this thread - but to add to the curiosity, i would much more like to understand (for the sake of more accurate statistics), is when a company is listed as running MS Exchange for example, are they actually "using" the product or is the "count" reflecting the purchase of a license that was never installed. On many enterprise agreements for both IBM and MS (as well as others) major companies pay a hefty fee for all products, and the mfg's add these license counts to their overall market share "count" irregardless if they are deployed or not.
