Contributed by phessler on from the hardware-is-what-you-kick dept.
Update! (from March 15th GMT) Only $2500 still needed!
Update #2 (from March 16th GMT) The goal has been reached! Thanks to everyone for helping out and making this a successful fund raising event!
Marco continues: "I will be driving the collection and purchase of the enclosure and drives. I'll set up a web page to track the donation status. If you'd like to make an anonymous donation just let me know and I will not print your name. Queries and paypal donations should go to slash at peereboom dot us. If you can't donate using paypal contact me off list and we'll work out a different payment method. If we raise more than the required amount I'll forward that cash to Theo.
The price of a fully populated PowerVault 220S with 4 hour on-site warranty is about $12500 USD including tax and shipping. Let me kick this off by donating $250 to this cause. A person that shall not be named donated a PERC4/DC RAID controller.
If OpenBSD is important to you this is the right time to step up and donate some of that hard earned cash. The folks in Europe can really make a difference due to the current Euro exchange rate. Let me stress that the CVS machine is the single most important infrastructure device for OpenBSD. The goal is to have this enclosure in Theo's hands before the hackathon so don't wait!"
If you want to help, paypal some money to slash at peereboom dot us, or you can use the OpenBSD ordering system. Be sure to mention its for the cvs machine.
(Comments are closed)
By crazythink (206.75.46.254) on
Comments
By Marcos Latas (81.193.182.89) carvalholatas@gmail.com on
By Anonymous Coward (208.252.48.163) on
I already do this in the form of paying for every single release rather than downloading it for free. Where's that money going?
Comments
By Anonymous Coward (64.235.239.2) on
Comments
By Anonymous Coward (69.143.29.120) on
By Syntax Error (81.204.188.152) on
Comments
By Marcos Latas (81.193.182.89) carvalholatas@sapo.pt on
By Fuu (62.175.42.214) on
Comments
By Christian Jones (66.92.35.242) chjones@aleph0.com on http://www.aleph0.com/~chjones/
And remember, as it's not included in GENERIC, OpenBSD software RAID is considered experimental. Not sure that's the attitude I want to take with what may be the most important data on the net. ;-)
Comments
By mirabile (213.196.227.95) on http://mirbsd.org/
as in OpenBSD (May 2004 at the moment) plus some fixes for "boot -a"
and autoconfiguration/autodetection support, and they've been doing
that for way over a year.
And I did this both out of space constraints, cost issues (who's
donating us? basically nobody, and others even burn t-shirts we
donate to them) and to make sure I'm not dependent on a specific
vendor's on-disc format.
I've sent in a couple of EUR just now.
Comments
By Samw (82.152.14.130) on
Comments
By mirabile (81.169.132.156) on http://mirbsd.org/
philosophy is all about. I just stated something about the
reliability of RAIDframe (for me).
I've even said that it's due to space and money constraints.
And you got something about the number of developers wrong.
By Anonymous Coward (66.131.206.88) on
Comments
By Lincoln DeCoursey (67.51.39.173) decourl () cs sunyit edu on
Lincoln
>Would be nice if some of the commercial Linux distro's would pitch in and >help for obvious reasons...
By ez (68.71.19.2) on
Comments
By Anonymous Coward (70.66.29.79) on
If you pay attention to what OpenBSD does and how its made you would relize you should trust these guys to know what they are doing. If you don't use OpenBSD why are you even here??
By Aasmund Midttun Godal (80.202.218.120) on
* Support
* Capacity
* Redundancy (hot-spare)
* Manageability (hot-swap, mgmt tools etc.)
* Performance (normal, and under load)
* Maintenanance requirements (maybe the developers prefer developing rather than spending time making sure CVS is up).
* Durability
I am sure you can find solutions which offer the same capacity and performance (which would theoretically be "good enough") for around $3k, however, the other points are very difficult to achieve with a tight budget.
By Anonymous Coward (68.123.254.41) on
The only place you'll see Dell after a donation is on the donations page.
By mirabile (81.169.132.156) on http://mirbsd.de/
By Anonymous Coward (69.197.92.181) on
Comments
By Anonymous Coward (70.66.29.79) on
Comments
By Anonymous Coward (69.197.92.181) on
By Oliver (62.178.151.180) on
http://www.expert-zone.com/index.php?module=announce&ANN_user_op=view&ANN_id=750
And yes, I already donated some money for the upgrade.
Comments
By Nate (65.95.240.35) on
Comments
By Oliver (62.178.151.180) on
No Theo & Team: No OpenBSD, no code.
No users: No OpenBSD, no money. Maybe a niche OS.
Conclusion?
Comments
By Anonymous Coward (69.143.29.120) on
Comments
By Anonymous Coward (69.197.92.181) on
By Anonymous Coward (212.143.248.152) on
Comments
By tedu (67.124.149.56) on
By Anonymous Coward (204.209.209.129) on
By Anonymous Coward (68.149.0.169) on
http://www.google.com/url?sa=U&start=1&q=http://www.feyrer.de/g4u/g4l.html&e=747
By morf (68.104.57.241) on
By Aasmund Midttun Godal (80.202.218.120) on
Comments
By Anonymous Coward (69.197.92.181) on
Comments
By Otto Moerbeek (213.84.84.111) otto@drijf.net on http://www.drijf.net
SMP is here because there was a party who wanted to have SMP, and they were able and willing to pay for development. It helped that NetBSD has SMP, so we could borrow code. It would have been silly to develop everything form scratch in this case.
By Anonymous Coward (81.64.227.144) on
So you're wrong: OpenBSD cares about users, and this money would profit users.
This would not be needed if devs didn't care about granting anoncvs access to you, damn user.
Comments
By henning (213.128.133.133) on
the replacement hardware is mostly needed due to
1) reliability concerns
2) performance problems
speed is a major issue here, and please, we know what hardware we need, SATA is far far far far slower with this usage pattern.
By Anonymous Coward (141.157.237.129) on
Comments
By Anonymous Coward (70.66.29.79) on
If you didn't have food in your fridge wouldn't you want to go buy food, before your belly is empty.
By Otto Moerbeek (213.84.84.111) otto@drijf.net on http://www.drijf.net
Read only access is different, there are a lot of anonymous CVS mirrors already.
The cvs.openbsd.org disk subsystem is a RAID system, but at this moment the RAID system itself is showing problems, while the load has been going up, since there are more active developers than before. The server itself does not need to be replaces, just the (very important) disk subsystem.
But as often, some people are very eager to give advice based on false assuptions, instead of trusting us to do a good job and supporting that.
Comments
By Lennart Fridén (213.64.159.220) on
Comments
By Otto Moerbeek (213.84.84.111) otto@drijf.net on http://www.drijf.net
But I do not like getting advice from peope that lack knowlegde (in this case about setting up a CVS repository) and just assume some facts and as a result are talking nonsense.
Marco is an OpenBSD developer, active in SCSI driver development. His day time job involves working with disks too. I trust his judgement on what kind of disk susbystem is best for cvs.openbsd.org completely.
By Has Theo endorsed this? (67.149.85.237) on
Comments
By Ben Goren (65.39.81.115) ben@trumpetpower.com on Ben Goren
See http://marc.theaimsgroup.com/?l=openbsd-misc&m=111058611324416&w=2.
Cheers,
b&
By Anthony (24.61.18.73) agabriel@home.tzo.org on
Comments
By Anonymous Coward (80.57.212.43) on
Since we're on the subject, what color do you want to paint the bikeshed?
Comments
By Anthony (24.128.97.219) agabriel@home.tzo.org on
Comments
By Anonymous Coward (69.197.92.181) on
Comments
By Aasmund Midttun Godal (80.202.218.120) on
Comments
By Sean Brown (68.147.202.55) on
Comments
By Aasmund Midttun Godal (129.241.211.73) on
Comments
By Anonymous Coward (69.197.92.181) on
By Anonymous Coward (69.197.92.181) on
By Anonymous Coward (203.10.110.133) on
Up until only recently, an IDE hard drive was only capable of servicing ONE command at any one time. While that command is being serviced, it could not even queue the next command. The IDE bus sits idle while the drive acts on the last command. The effect is a duty cycle in the drive and another seperate duty cycle on the IDE bus. While one is active, the other is idle.
Now look at modern SCSI. A SCSI drive can queue up to 256 commands and while one is being serviced, the others can be sorted within the drive to exploit elevator sorting techniques. Drastically reducing head movement. All at the same time, SCSI can allow a drive to service a command, sort queued commands and allow commands to traverse the SCSI bus.
A SCSI drive which is a little slower than an IDE drive on paper, can end up being MANY TIMES faster at high I/O tasks like web and database serving. And hey, CVS servers.
Just now, some SATA drives allow up to 32 commands to be queued and I think sorted, but this requires controller or driver support.
By henning (80.86.183.226) on
people, please. we run that machine for a while. we know where its performance bottlenecks are, and what the usage patterns are.
I mean a sata controller with a large /very large cache can easily do this.
this statement is incredibly wrong.
Comments
By Aasmund Midttun Godal (129.241.211.73) on
Comments
By djm@ (212.198.106.26) on
By mirabile (81.169.132.156) on http://mirbsd.org/
That point comes each time, because it's about
the only thing _pro_ IDE.
They know why they're using SCSI.
By Anthony (68.145.103.21) on
By Marco Peereboom (68.82.91.100) slash@peereboom.us on http://www.peereboom.us
I know this might sound funny to some but please just trust our judgement on hardware selection. I appreciate the suggestions but we really need higher end equipment in this case.
EBAY is not an option because we need the warranty if something does go wrong.
Thanks and keep those donations comming!!
/marco
Comments
By Nikolas Britton (12.223.129.46) nbritton@nbritton.org on
OpenBSD is "low cost". Is OpenBSD "low end"?
Comments
By Marco Peereboom (143.166.255.18) slash@peereboom.us on www.peereboom.us
Comments
By Anonymous Coward (71.0.126.14) on
Sorry, but I don't trust your judgement. You still haven't justified a $12k expense on this. What part of "justify" do *you* not understand? Why must it be this expensive ass piece of hardware when there are other, just a capable, and less expenive options available to you.
I want a Porsche Carrera GT to get me about, but a Kia Rio does the same job just as well for a fraction of the cost. It may not get the oohs and ahhs but at least it gets me there, and I don't have to remortgage my house to get one.
It comes down to this. There's what you want, then there's what you need. If the CVS server is in danger, then what you need is an economical replacement that you can get now and that works now. Try asking, someone may donate some working RAID solution to you. It may not be what you want, but it is what you need and it works.
Now what you want is the Carrera GT of RAIDs, and well... you can want in one hand and shit in the other... which one do you think fills up faster?
Until you give some compelling reason *why* this damned contraption is the only option availbale to you, many of us, including myself, won't give you a dime for it. Can you say "misappropriations"? Your coming across as tantrimic child who wants the Deluxe Transforming Ultra-MegaLords Action Playset (with real working lights and sound! (batteries not included)) who's having a hissy because his parent says they can't afford that right now.
Beggars really can't be choosers. The following pages might help you understand the dilemma that many of us are facing in light of you request:
http://dictionary.reference.com/search?q=need
http://dictionary.reference.com/search?q=want
Comments
By morf (68.104.57.241) on
Comments
By Anonymous Coward (71.0.126.14) on
Comments
By morf (68.104.57.241) on
if you don't can't/won't donate thats fine, just don't pretend you were ever going to.
By henning (213.128.133.133) on
well, there are no just as capable, less expensive options available. from repeating that there were it doesn't become true.
Comments
By Anonymous Coward (71.0.126.14) on
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us
And thanks for your links to dictionary.com to explain need and want; they were refreshing.
Comments
By Anonymous Coward (71.0.126.14) on
Maybe your proximity to the project makes you think you know it all, but I delude myself into thinking that the OpenBSD camp is a bit more knowledgeable and resourceful than most other netizens out there. The collective of the user base has proposed lots of lower cost alternatives on this very site and you have arrogantly ignored them, intent to spend come hell or highwater. Many of these people work in IT and have to make thier systems work with the same level of demand but on a shoestring budget. If you want to continue to act like a know-it-all in the face of these individuals, then don't expect much sympathy when people don't contribute beyond the semiannual CD sales. (Yes, I know CD sales are ailing).
I am proud to have stood up and have asked why, for suggesting that you look into more economical alternatives instead of just whipping out my checkbook right away. I am ashamed that you saw such a simple question as beneath you to answer and choose to respond to it with venom.
It would be sad to think that OpenBSD's needs might suffer because of your arrogance.
Comments
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us/pv220s_quote.pdf
See the quote at: http://www.peereboom.us/pv220s_quote.pdf Note at the bottom the $11774.49 grand total. Whenever such a unit is purchased it is good practice to also purchase an additional drive as a spare. At $549 a pop we are now at 12323.49. Also note the no shipping costs and let me tell you how it works for a box that weighs over 60lbs. It'll be easily $200. So the total of having the box in my hands is already $12500+ and then it needs to make it to Theo's, not to mention money transfer fees and other assorted fees that'll magically pop up when dealing with such an effort.
Oh and you too can get a quote like that by surfing to: http://www1.us.dell.com/content/products/compare.aspx/storage_store?c=us&cs=04&l=en&s=bsd
and clicking some buttons. Now that you know how I came up with the magical number it is ok for you to disagree with our selection of hardware. You are however unaware of the IO load on the box and you project your view of reality upon the decision that was made by some pretty smart engineers who do this for a living. You might find this funny, I find this infantile, disrespectful and insulting. Like most OpenBSD developers I have a real job with real demands. I give you my time and expertise for free and you thank me with insults. Trust me, I can do without.
I donated $250 and I’ll end up with most of the fees. What have you done besides insulting OpenBSD?
Comments
By Anonymous Coward (71.0.126.14) on
"Since you are out to make me look like an ass I'll disregard your insulting comments ..."
You did that for yourself. I just pointed it out to you. Don't shoot the messenger. If I really wanted to have at you, I wouldn't have bothered being so long winded. The actual request was really quit simple and did not merit confrontation.
"... and be the nice guy by showing you how I came up with the magical $12500."
Isn't that all that was really asked for? What exactly are the contributors really getting OpenBSD for thier money? You might want to link all of this to the main page so anyone else who has questions can see where thier money is going.
"You might find this funny, I find this infantile, disrespectful and insulting. Like most OpenBSD developers I have a real job with real demands. I give you my time and expertise for free and you thank me with insults. Trust me, I can do without."
There is an old saying: "Never trust a programmer with a screwdriver." Just because I trust the coding of the OpenBSD core team doesn't mean I trust their other descisions. Your descions are not infallible. And I don't find it funny at all. I find it sad that as a representative of OpenBSD you decided to be ornery with this disclosure. Though showing us how you arrived at these numbers is a huge step, the core question is what does this solution bring to the table that cheaper SATA RAID solutions do not. Many have lampooned the SATA idea, noone has actually botherd explaining why it is insufficent here (that I have seen). Not being a RAID guru, a quick google shows me numbers that reflect SATA RAID performance in a better light, and equivilent redundancy. There are many opinions on the matter, I want the facts.
"What have you done besides insulting OpenBSD?"
I recall saying nothing derogatory about OpenBSD. Please don't project unimplied emphasis upon my words. My dissatisfaction is with your lack forthcoming as to why a more cost effective solution was not viable. No more, no less. "Trust me" is never a sufficent response.
Comments
By Anonymous Coward (202.45.125.5) on
Unfortunately, computer science being industrialized, has caused the move away from "computer scientists" and moved towards a split between programmers and computer engineers.
People who design Operating Systems, I would like to think know intimately about hardware, software and their efficient interaction.
Worse still, on an unrelated note, API's are removing the programming from programmers and bringing non-programmers to programming.
By Anonymous Coward (69.197.92.181) on
Comments
By Anonymous Coward (202.45.125.5) on
Seagate SCSI and SATA 3.5mS versus 8mS.
Notice all the SATA drives falling under the "Desktop" category?
Western Digital SATA 4.5mS. Fast, but also expensive and shown to not match the high end SCSI drives at high I/O, even with TCQ enabled. I would like these in my desktops, but then, I'd like the Maxtor SCSI 320's in my desktops more. ; )
Fujitsu SCSI 3.3mS.
Maxtor SCSI 3mS.
Maxtor SATA 9mS.
I made a slight mistake before linking to a storagereview page. The one with all the SCSI 320 drives and the Raptor SATA coming last. I accidentally clicked on the 'Server - Capacity' criteria, instead of the 'Server - Performance' criteria. Funny though, the SCSI 320 drives meant for 'capacity', actually outperform the SATA drive meant for 'performance'. Take a look at these, side-by-side, to rub a little salt into your ego. 5.5mS versus 8.1mS and 97-74Mb/S versus 72-54Mb/S transfer rates.
Hell the SCSI drives transfer rate at it's slowest, is faster than the SATA drive at it's fastest.
I love OpenBSD. I would love to see a core development server for this fine OS with nothing but the best components for the job. The developers have to use it and I am fine with their decision to go with high end performance and safety. Why do you care so much? Are you really caught between wanting to donate and being dead set against the spec? Are you just trolling?
PS, do you really believe MTBF and MTTF figures quoted by companies that are trying to sell you something? One of the few specifications that we can't really accurately test? 1.5M hours MTBF is 171 years. Do you really believe that crap?
Comments
By Anonymous Coward (195.85.225.142) on
By Chris (24.76.170.207) on
And that says it all right there. You don't trust their judement to have thought out, and have a good idea of, what they need for a CVS server, but you trust their judgment to provide you with an OS that (I assume) is critical to something you do on a network somewhere? Maybe you can provide me with a dictionary.com link for cognitive dissonance :)
It seems like all the SATA suggestions mostly look at cost and disk space alone. Neither of those seem like the primary concern for the CVS server that makes OpenBSD happen. This isn't a big fileshare or music server handing out relatively large files to the Internet.
I'm glad you're proud of having "stood up and questioned" people on the Internet... very brave.
Comments
By Anonymous Coward (71.0.126.14) on
Comments
By tedu (67.124.149.56) on
seek time (among a few other things)
Comments
By Anonymous Coward (71.0.126.14) on
Comments
By Bruce (24.85.90.11) on
By CyBoon (165.21.154.116) on
Because your questions are about nothing and achieve nothing except to
make you feel good about yourself. At the end of the day, these folks
will use what they ask for. They will take responsibility, as they have
always done, all these years. Are you prepared to do the same? To make
the same effort and sacrifices?
I am amazed at how nicely marco has treated you. Despite your vulgar
and uncivilised behaviour. I stare in amazement at your questions,
this one in particular:
"What exactly are the contributors really getting OpenBSD
for thier money?"
Isn't it obvious? No? Every second an OpenBSD machine stays up,
every attack they weather, every hour saved in deployment, every
tiny detail taken care of is worth every cent I contribute.
Please don't 'stand out and ask questions' on my behalf. I am
an OpenBSD user and I have much higher standards than you ever
will. I have asked hard questions from my OpenBSD systems every
day for the past 6 years, as many other OpenBSD users have.
Many of us users are serious folk with hard problems to solve.
Your 'problems' with OpenBSD mean very little to us. At the end
of the day, they deliver. That's all.
The OpenBSD folks have been doing good work for a long time.
It is clear from their track record (code and operations)
that they know what they are doing. Did you give them $12K?
Did you volunteer to help out? Do you even have the skills
to do so?
There are people who get things done and then there are the
folks who are 'concerned' about this and that. Who 'stand up
and ask questions'. Who will ask questions for the sake of
hearing their own voice and 'participating'. You are clearly
in the latter group.
Unlike you, I have a lot of respect and trust in the OpenBSD
folks. I greatly appreciate all the hard work they've put in
all these years and big-talking posers like you who think
they're good enough to talk like that the the OpenBSD crew
simply makes me furious.
Comments
By Anonymous Coward (71.0.126.14) on
So far this dialog has only made me think less of certain members of the OpenBSD community because of the arrogance and self-importance placed on display over such a trivial query. What you, I, or anyone else thinks of themsleves, thier contributions, or OpenBSD in general, has nothing to do with the capabilities of the hardware in question versus the more economical proposed solutions. Instead of answering the question, many of you have chosen to hold an inquistion. Remind me again, is this an OpenBSD website or a papal rally? And before you point that finger asking what have I done, what have you done?
So far only two people have actually posted legitimate answers, and neither of them were Marco. And for all your pompusness just now, it sure as hell wasn't you either. You just jumped on the bandwagon of people making excuses why for the question shouldn't be answered, and how your trust should be sufficient to merit my donation.
I asked nothing on your behalf. You are not the skeptic, you are just a sheep willing to go along with whatever is asked of you. You were free to continue grazig in your pasture, but instead decided to speak out here. You are obviously willing to pay top dollar for eveything you buy when the brochure says top-of-the-line, regadless of whether or not more cost effective and capable solutions are open to you. C'est la vie. But don't expect everyone to be just like you.
Also, you, like so many other of the less astute posters here (not all the posters, just the elite few), have chosen to put words into my mouth about OpenBSD. Nothing I have ever said had anything to do with the quality or capabilities of OpenBSD. In fact, I have said the opposite. But that soapbox you are standing on seems to have put your head that much farther up your ass and rendered you incapable of seeing that.
Concern for how OpenBSD makes use of the funds it is asking for can mean the differnce between paying Dell a thief's ransom for superflous crap or supporting the next piece of non-donated hardware. If that's not a concern of yours, then that would be your shortcoming. Don't play holier-than-thou to those who do.
Everyone here wants what is best for OpenBSD, and that includes getting the best bang for the buck. I proposed no reason for or against the SATA RAID, I only asked why this was not an option here as it appears quite capable and much less expensive. You are the ones who decided that is was blasphemy and that you are somehow better than I because of it. This speaks more of you than any slur I could ever cast your direction.
And despite what any of you think about respect and trust, I can disagree with a person's decisions or attitude without loosing faith in that individuals capabilities, or respect for everything they have done before and after. I respect everything Marco Peereboom, and every other contributor has done for OpenBSD. But that's a far different cry from the absolute faith many of you sychophants have put into them and on display here in the past few days. As lemmings go, you really have nothing to be proud of.
Comments
By CyBoon (165.21.154.108) on
By bsdguy (69.45.74.35) on
Comments
By Nate (65.95.243.105) on
It's right there man.
Comments
By Anonymous Coward (68.149.0.169) on
Come on, this is OpenBSD, and we've seen great progress from them. This isn't one of those "goodwill" Tsunami donation drives that does basically nothing for the people, except to garner more publicity for donations.
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
Comments
By Tobias Weingartner (68.148.1.194) weingart@tepid.org on
> read pass the first line but, "JBOD with U320 SCSI", First of all JBOD is
> NOT Raid!
Well, that depends entirely on where your RAID functionality is located.
If the RAID card happens to be in/on the motherboard, a JBOD enclosure
will be exactly what is needed. No?
> and in no way do you need U320 SCSI because your bottleneck is your
> connection to the net and to a lesser extent your system.
Really? What does the CVS server do when you do a large checkout/update? If you research this particular issue, you
will find that the issue is not necessarily network throughput,
and having your disk subsystem keep up with network, but it is
quite simply disk I/O.
> maybe if you
> had a 10Gbit (1.3GB/s) link but I doubt you even have a 1Gbit link
> (130MB/s, PCI is 32-bit/33Mhz = 130MB/s, PCI-X 64-bit/66Mhz = 530MB/s,
> and PCI-X 64-bit/133Mhz = 1GB/s) to the net.
What makes you think that the local network the main cvs server is on
does not have at least Gb networking connectivity?
> I don't see how you can
> justify $12,500 for a JBOD U320 array when I (and you) can build a 4TB
> SATA hardware based hotswap (trays, racks, chassis, everything!) RAID5
> system or even a hybrid SCSI/(S)ATA for 1/4 - 1/3 the cost and and then
> still have a shitload of money after you buy that near-line remote (wifi
> etc. in another building) JBOD 4TB ATA system to backup the 4TB raid
> array. I've spent the better part of a year researching and prototyping
> (on paper) low cost large raid systems and id be happy to share it with
> you.
Yes, I'm sure you did, for your needs. Personally, I doubt that cvs
needs terrabytes and terrabytes of storrage. It is about I/O throuput
and a particular type of disk access pattern. However, I was not part
of the group that put together this particular spec. If I was so
inclined to waste those people's time to find out the what, why, and
how they came to the conclusion they did, chances are pretty good that
I would come to a very similar solution.
When it comes all down to it, you're talking to people that have been
doing this professionally for a *LONG* time. We, if anyone, know what
our resources and limits on those resources are. We are the ones that
are the developers of this system. If you trust us to keep your data
safe, to keep security in our minds always. Why would you not trust
us when we say that we need this? Because we want to "play" with a
U320 scsi system? Trust me, many of us have worked with many such
systems in our work, or past lives. Asking for money, to get such a
toy to play with, would be the last thing on *MY* mind. For $12'500
I could pay some of my bills, go on that honeymoon in style, and in a
timeframe that would be more fitting than "sometime when we have money".
--Toby.
By Anonymous Coward (203.10.110.131) on
You think that the only conections this machine will be making, will be from the net?
I am not even an OpenBSD developer, but I realise that this server probably has builds for multiple architectures happening very often from machines that are connected within the same LAN as it (snapshots? An integral part of QA), as well as people from the net updating their own trees. I would imagine that it would be busy with loads of high I/O requests.
Do you want SATA for high I/O? Or would you prefer the disk subsystem which has been specifically designed with high I/O in mind, SCSI?
Please share your past years, on-paper experience with these people which understand these systems at very low levels, write drivers and deploy in the real-World.
PS, I don't have a credit card, what is the best way for me to donate? Can I direct debit?
Comments
By Marco Peereboom (143.166.226.19) slash@peereboom.us on www.peereboom.us
Comments
By Anonymous Coward (213.118.35.44) on
Comments
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us/
Thanks!! /marco
Comments
By Anonymous Coward (213.118.35.44) on
By Anonymous Coward (203.10.110.131) on
On second thought, after looking at your web page, complete with your favourite colours and very wrong in places "DOS to UNIX" reference, I think the devs can do without hearing about your storage "research".
label = disklabel?
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
That DOS to Unix reference (and most of the FreeBSD page) was from when I was first starting out with FreeBSD and the UNIX command line, it's not even complete. The thing you have to remember is that my site is mainly for me to write my scribbles and notes down.
My history is with DOS, Novell, Windows and a little bit of OS/2 not with UNIX. I am not a developer and have never said I was one, but I'm learning. My expertise is in hardware, systems, and network administration / integration / engineering or whatever you want to call it and I do clam to know a thing or two about hardware because it's what I do for a living, and I love doing it.
"label = disklabel?"
you are %100 right, but at the time I did not know that.
Comments
By Anonymous Coward (202.45.125.5) on
I've been in electronics and computing since 1989 but I am just a humble long-time OpenBSD user. But I realise that hardware goes hand-in-hand with software.
You cannot fully understand the best hardware fit for an overall system, without knowing the loads placed on that hardware by the software. It is the software choice afterall, which determines the hardware needs.
If you are choosing software based on the hardware you have, you have put the cart before the horse. I don't go out and randomly buy the fastest hardware, only to later find that I will be limited to Microsoft Windows products. I choose my software (OpenBSD, Mac OSX, db, www, smtp, nntp, pf) and then spec the hardware accordingly.
I believe that SCSI is by far the right decision for a busy cvs server (or most any other server for that matter). However, if the devs chose otherwise, I would realise that I am busy with my work and have not put the time into considering what has been on the plate before them.
I might ask why they decided to go with their decision, out of curiosity to learn, but certainly not bark at them about how wrong they are after I've thought about it for a few minutes. Yes, I'm aware you've spent the better half of the last year spec'ing out storage for the need before you, but you do not have all the criteria that these OpenBSD devs do before them.
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
By Matt Healey (203.31.101.76) on
It sounds to me more like you want U320 to play with it. Not because you need it. Even a quick glance around shows up better, cheaper options.
What about an XServe RAID?
The mid priced unit gives you 2.8TB across 7x 400GB disks, Dual RAID controllers with 512MB's of cache each, dual power supplies and 3 year, 4 hour on-site replacement all for $9,500. That still leaves you 7 drive bays free for further expansion.
If something like that is good enough for Cisco and Oracle, and also good enough for Apple to run the iTunes Music Store, I fail to see why it would not be good enough to run the Open* CVS site.
Comments
By Nate (65.95.243.105) on
Comments
By Matt Healey (203.31.101.84) on
Comments
By Nate (65.95.243.105) on
Comments
By Duffman (82.122.166.110) on
Comments
By Anonymous Coward (213.84.127.221) on
Comments
By Duffman (82.226.59.105) on
By Marco Peereboom (143.166.255.18) slash@peereboom.us on www.peereboom.us
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
LSI Logic MegaRAID 320-2 (PERC4/DC): $Free
15 ACARD ARS-2100Q Ultra160 to IDE Bridge Adapters: $1335, mars-tech.com
15 250GB 7200RPM IDE Hard Drives: $2,000
SCSI Cables and Connectors: $300
4U 9-bay Rackmount Storage Chassis: $500
3 Chenbro 5.25" 3-bay to 5 3.5"-bay Hotswap IDE backplanes/cages: $600
-----------------------------
Raid Array Specs:
15 250GB IDE Drives.
Raid 5 Capacity, minus one hotspare and parity drive: 3.2TB.
Max Throughput: 320MB/s (two Ultra160 channels).
New CVS Server: $1430
Intel E7210 Server Motherboard "SE7210TP1-E", Boxed: $210
Intel P4 3.0E GHz 800MHz FSB, 1MB L2 Cache, HTT, Boxed: $180
Kingston Dual Channel Kit 184-Pin 1GB(512MBx2) ECC Registered DDR400: $350
Keyboard, Mouse, Monitor, floppy: $170
2U Enermax rackmount case: $230
LG 52X32X52 CD-RW Drive, Model GCE-8526BB: $24
2 Western Digital Raptor 74GB 10,000RPM SATA Hard Drives $260
Recreate Old CVS Server into new "Near-Line" (Mirror of Raid 5 Array) Backup Server: $2260
2 HighPoint RocketRAID 1640's SATA Controllers: $180
10 300GB SATA Hard Drives: $1900
13-bay ATX server case: $140
802.11b/g PCI (Atheros chipset) Adaptor: $40
Misc: $2600
Tripp-Lite Rackmount LCR2400 AC Line Conditioner: $360
Tripp Lite Smartpro 5000 UPS 5000VA, 4000 Watts, 3U Rackmount: $2,240
Grand Total: $11,030 and I wasn't even trying to cut costs down (just ballparked everything), lot more bang for your buck!!!
Comments
By Anonymous Coward (128.114.59.123) on
Comments
By Nikolas Britton (12.223.129.46) nbritton@nbritton.org on
Thats the whole point of a RAID. when you have hotspares, hotswapping and don't deliberately run the RAID in degraded mode then the MTBF for hard drive failure is null. The only thing that will stop you is an "act of god", a failure in other parts of the system, or a power failure and to counteract those you can buy 2 SATA servers (mirroring them, RAIP) a UPS and a generator for the price of an equivalent SCSI server. Now the only thing that will stop you is an "act of god". to counteract "acts of god" you can buy a l-cheap-o PC and mirror the array from anywhere in the world. now the only thing that can stop you is the apocalypse. but by then it doesn't matter :-).
By henning (80.86.183.226) on
By Anonymous Coward (144.136.95.144) on
Comments
By Marco Peereboom (143.166.255.18) slash@peereboom.us on www.peereboom.us
By zyphr (83.68.2.195) on
By Anonymous Coward (164.86.99.3) on
Comments
By Anonymous Coward (208.252.48.163) on
Comments
By Anonymous Coward (199.67.140.77) on
Not all CVS activity consists of commits.
By Anonymous Coward (24.12.6.251) on
There is a reason that HDS and EMC are offering SATA drives in their enterprise products. Those guys don't mess around - their reputation is on the line. If SATA sucked so bad, they wouldn't be doing it. (And, yes, I know their controllers do some extra things for the SATA drives, etc... but it's irrelevant here.)
These guys offered valid points and got bashed. It's easy to see why the OpenBSD crowd has a reputation for thinking their shit don't stink.
By Anonymous Coward (67.102.173.11) on
Comments
By SH (82.182.103.172) on
One may enjoy a well written troll post (very scarce, I'm afraid), but this is pathetic.
By morf (68.104.57.241) on
Comments
By morf (68.104.57.241) on
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us
Subject: cvs.openbsd.org needs an upgrade, a few days later
Date: March 14, 2005 5:50:47 PM CST
To: misc@openbsd.org
First off I want to thank all people that donated money. Many people sent small and big donations and we are about $2500 away from the requested amount. So please keep those donations coming!! Your donations will have a direct impact on the continuity of the project.
The surprising thing in this whole experience is the lack of donations from companies. By far most donations came from individuals who care for the project. If you work for a company that uses OpenBSD code please urge them to make a donation so that we can continue to deliver quality software for free.
I challenge the community to step up and get the last few bucks in so that we can order the equipment before the end of the week!
To all who already donated thanks again :-)
/marco
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
Triple Redundanty 760W Power Supply
8 Hot-swappable Drive Bays
6 80mm 5000RPM fans
13x 74GB Western Digital Enterprise SATA Drive: $180 ($2,340)
(8 for Raid Array, 2 for RAID1 System Drive and 3 Spares)
5-year warranty and 1.2 million hours MTBF
Optimized for Server Access Patterns.
Average Read Seek Time 4.5ms
Average Write Seek Time 5.9ms
Command Queuing
10,000 RPM
2x Intel Boxed P4 Server Platform: $1,140 ($2,280)
Intel Boxed E7210 Chipset Server Board "SE7210TP1-E": $210
3 64-bit/66MHz PCI-X Slots
Intel Boxed Pentium 4 3GHz Processor: $180
800MHz FSB
1MB L2 Cache
Kingston ECC DDR400 1GB(512MBx2) RAM Kit: $350
Misc. Server Parts: $400
2x Your Choice of SATA RAID Card: $1,000 ($2,000)
1x TrippLite Fault Tolerant Bypass 3000VA Online UPS $1,100
Grand Total: $9,280 ($6,360 for 1)
$3,220 Dollars Cheaper and Your Buying an Extra Server, RAID Card, and Chassis for "On-Site" Replacement Parts Plus a UPS System... Oh yea, I forgot to add in the cost of a 3000 Watt Generator, Tack $700 onto the price and an Online JBOD Mirror of the RAID Array, Tack on Another $700. I can't see how you can find fault in this system because everything is fault tolerant because your buying two systems, one for cannibalizing when you have failures on the main rig. And the Hard Drives are Ultra160 SCSI Drives with SATA Controllers. The SATA spec supports roughly everything the SCSI spec does, notably Command Queuing and Hot-Swap. And your getting 2.2 times the size of your current array.
Umm, There It Is, Good Day.
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
Comments
By tedu (64.173.147.27) on
where are the 15k sata disks?
where are the 15k sata disks?
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
I'm just trying to save you guys money in the long run...
Comments
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us/
Don't you think that the follow on technology isn't going to be SAS?
Ever heard of FC btw?
There are many technologies available. SATA represents the low end technology; its good for some things and bad for others. RAID is one of those technologies that require intelligent drives. SATA drives are meant for desktops and simply lack the sophistication to be good devices in a RAID environment. Besides they are used to test durability and by being on the edge of technology they are less reliable. There is a reason why you pay a premium on SCSI and there is also a reason why it exists.
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
Don't you think that the follow on technology isn't going to be SAS?
Ever heard of FC btw?"
You are under the mistaken belief that you are an enterprise and that you need enterprise grade equipment. You (the project) are no where close to enterprise demand or requirements and saying that you need SAS and FC for what is essentially a file server makes you look stupid, please get back in touch with realty and the scope of the project.
"There is a reason why you pay a premium on SCSI and there is also a reason why it exists."
Wouldn't it be smarter and better to split the CVS trees onto 2 or 3 low cost commodity servers then to dive into the high end and premium priced equipment just so you can keep everything on one server and play with high-end equipment? This way your CVS server won't take a shit when one of your Non-RAID JBOD SCSI disks fail.
From you actions and comments on this thread you come of as not having though this thought, arrogant, and that your logic is skewed.
Comments
By Anonymous Coward (12.223.129.46) on
Comments
By Anonymous Coward (202.45.125.5) on
While you are pointing people to helpful dictionary.com URL's, you might like to look into the word "than".
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
By Anonymous Coward (202.45.125.5) on
Have you ever logged into that cvs server to analyse its performance characteristics? You are under the mistaken belief that you know systems design better than people who have been developing Operating System software and deploying and tuning it with hardware considerations for years. Your "help" is embarrasing. What would anyone outside of the developers of this project know about this project and its requirements?!
Here you can see a test that might not be equal, would first appear to be biased towards IDE, but most importantly shows that specs on paper mean little when a device is not being used as it was intended to be. It's an old test, but validates my point. The IDE disk is about 50% faster with sequential transfers from disk, than the SCSI disk but the SCSI disk has about 40% faster seek time. But how would you explain, given these numbers, the SCSI disk, that is three years older than the IDE disk, being about SEVEN TIMES FASTER than the IDE disk with a high I/O task? Inbuilt queuing and elevator sorting.
SATA is designed for desktop use. It leans towards high transfer rates for single use. Newer SATA drives with TCQ allow for higher I/O use for the increasing stresses multitasking puts on disks with modern desktop operating systems and applications. But still SATA TCQ (for desktops) uses 8 times smaller queues than SCSI TCQ (for servers). Oh shock horror! Expensive SCSI design leans towards servers and cheap SATA design leans towards desktops! What is the World coming to?!
That, Nikolas, is called being practical. And that is partly why SCSI still exists. I remember when people were proclaiming SCSI would soon be dead because IDE now allowed DMA transfers and transfers of more than one sector per interupt! People would do transfer rate tests that showed IDE doing a little better than SCSI and then seperately do seek time tests that were close to SCSI and claim that SCSI no longer had a great lead over IDE. Those ignorant people failed to realise the real draw card of SCSI. A SCSI drive could execute a command, while queuing commands from the SCSI bus and sort them for reduced head movement, all at the same time. These people doing dumb tests that had drive heads going start-end-start-end-start-end with a sythetic test that forced that to occur, failed to ever realise that a SCSI disk faced with such a scenario in the real World without having this forced onto it, could end up going start-start-start-end-end-end much faster.
SCSI320 running faster than a Raptor with TCQ enabled in a scenario perhaps similar to what a busy cvs server might expect. Notice this fastest SATA drive last in this small group of SCSI drives?
This way your CVS server won't take a shit when one of your Non-RAID JBOD SCSI disks fail.
You can RAID the disks in a PowerVault 220S by using a RAID card like, oh I don't know, a PERC4/DC RAID controller? Like the one which has been donated?!
From you actions and comments on this thread you come of as not having though this thought, arrogant, and that your logic is skewed.
You have exhibited a complete lack of understanding of what really matters with this cvs server. You are focusing on numbers while ignoring the biggest problems that face something like a cvs server. Lots and lots of requests for lots of small amounts of data that are scattered all over the place. SATA, designed primarily with sequential transfers in mind, does not do as well as SCSI for this task.
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
No, I'm not a developer and therefor have no need for CVS except for downloading source code. from my perspective, as a downloader, it works no different then an glorified FTP server.
"You are under the mistaken belief that you know systems design better than people who have been developing Operating System software and deploying and tuning it with hardware considerations for years."
I think I was a bit to harsh in my reply to marco, sorry for that... but being an excellent software developer does not automatically make them an excellent systems administrator, integrator, or engineer, vice versa. I don't want to say that I'm better them him, I don't know him, but It feels like he has an agenda (high-end and costly SCSI system) that is not inline with what the project needs, want and need are two different things.
"Your "help" is embarrasing. What would anyone outside of the developers of this project know about this project and its requirements?!"
Your right, I don't fully know what the requirements are because I have nothing to do with the project. But from what I do know is the current server is a Pentium 3 with 1GB of RAM and is using 14 18GB Ultra160 SCSI drives in a RAID 5 configuration. Based on the specs I do have I could and will speculate that it is an off the shelf uni-proc P3 and that it has no PCI-X slots, only PCI, and that it's LAN connection is only 100MBit FDX. You can only go as fast as your slowest link and that is the LAN connection, the PCI bus, the memory bus (and how much you have), and lastly then CPU. The PCI bus is limited to 130MB/s and the LAN link is 12.5MB/s each way. I don't fully know what happens in a CVS repo/server but I suspect that it is similar to ftp, http (no dynamic content), nfs, or smb file server and these servers are always network bound and when not networked bound, say for example 1GBit and 10GBit links, they are bound by the PCI bus. A single 5400RPM IDE drive can saturate a 100MBit link and a 1GBit link can saturate the the PCI bus. Based on the system spec that where stated above It could be assumed that this system will not even be able to utilize Ultra320 to it's full potential because of the other system limitations.
"But how would you explain, given these numbers, the SCSI disk, that is three years older than the IDE disk, being about SEVEN TIMES FASTER than the IDE disk with a high I/O task?"
This has to do with SCSI's command queuing and low read/write times do to high RPM drives, 10,000+, with non-sequential / random access patterns SCSI will always beat IDE because IDE doesn't have ether of those features.
"SATA is designed for desktop use. It leans towards high transfer rates for single use. Newer SATA drives with TCQ allow for higher I/O use for the increasing stresses multitasking puts on disks with modern desktop operating systems and applications. But still SATA TCQ (for desktops) uses 8 times smaller queues than SCSI TCQ (for servers). Oh shock horror! Expensive SCSI design leans towards servers and cheap SATA design leans towards desktops! What is the World coming to?!"
SATA is designed for anything you want it to be. It is only a bus interface, like SCSI, and what really matters, both in SATA and SCSI, is what's connected to it. With drives such as the SATA based WDC Raptor it can be a valid competitor against SCSI, at this point only Ultra160, because the SATA interface supports most of the features that make the SCSI interface the niche product it is today.
P.S umm I not sure which SATA drive your talking about... but I'm basing all of my comments off of the new WDC 74GB Raptor, not the 36GB Raptor or any other SATA drive.
"That, Nikolas, is called being practical. And that is partly why SCSI still exists. I remember when people were proclaiming SCSI would soon be dead because IDE now allowed DMA transfers and transfers of more than one sector per interupt! People would do transfer rate tests that showed IDE doing a little better than SCSI and then seperately do seek time tests that were close to SCSI and claim that SCSI no longer had a great lead over IDE."
SCSI is not dead nor is it going to die anytime soon. It has it's niches, especially in the high-end and big iron markets. Those people obviously forgot to do random access pattern tests and again this has to do with the SCSI vs IDE bus. IDE will always loose when you compare this aspect. But with SATA-I, SATA-II, and SATA-III standards you will see in the coming years that it will dominate the entry-level server market and will have a sizable market in mid-level servers. This is do to simple economics and the reasons stated above (and below), you can quote me on that.
"Those ignorant people failed to realise the real draw card of SCSI. A SCSI drive could execute a command, while queuing commands from the SCSI bus and sort them for reduced head movement, all at the same time. These people doing dumb tests that had drive heads going start-end-start-end-start-end with a sythetic test that forced that to occur, failed to ever realise that a SCSI disk faced with such a scenario in the real World without having this forced onto it, could end up going start-start-start-end-end-end much faster."
Agree with you, see above. One thing many seem to be forgetting is that it's the job of a good, one that has has a real I/O controller, cache, etc., RAID controller to play traffic cop to efficiently sort and move data to and from the disks. This lessens the need for command queuing and low access times on the drives if it is done correctly.
"SCSI320 running faster than a Raptor with TCQ enabled in a scenario perhaps similar to what a busy cvs server might expect. Notice this fastest SATA drive last in this small group of SCSI drives?"
? maybe it was in that link you gave me, sorry didn't read it (yet)... Ultra320/15K vs SATA-I/10K, no contest, SCSI will win all of the tests.
"You can RAID the disks in a PowerVault 220S by using a RAID card like, oh I don't know, a PERC4/DC RAID controller? Like the one which has been donated?!"
You can and it does but he stated that he wants to use it in a JBOD array configuration and that is just wrong. If one and only one of those drives failed it would take the entire array with it. JBOD is not RAID.
"You have exhibited a complete lack of understanding of what really matters with this cvs server. You are focusing on numbers while ignoring the biggest problems that face something like a cvs server. Lots and lots of requests for lots of small amounts of data that are scattered all over the place. SATA, designed primarily with sequential transfers in mind, does not do as well as SCSI for this task."
No I haven't. SATA paired with the right drives can do just about anything Ultra160 can. The main reason that I see for him wanting Ultra320 is that his Ultra160 backplane is dieing and that his Ultra160 Array is just about maxed out and he's using it and the $400 donated Ultra320 RAID card as an excuse to ask you for $12,500 so he can have his Ultra320 Array. This problem is easy to overcome with other cheaper cost effective solutions, SCSI or not. One of those could be splitting the CVS trees onto separate servers. Another option could be splitting the single RAID array into multiple separate RAID arrays by using more then one RAID card in the system and split the trees over the arrays, SATA is a good candidate for both of these solutions. In my day job if I came to the boss, well I don't have a boss as I am the boss so owner of the company, and said I need $12,000 for a new foobar and not justify it or justify why nothing else will work you know what will happen. I will be shot down - no questions asked. My bullshit meter is ringing like crazy and thats why I'm "putting up a fight" with his request. His comment about SAS (Serial Attached SCSI) is troubling too as to my knowledge they are incompatible buses because one is parallel the other is serial (just like ATA is to SATA) so what's he going to do when he's bored with Ultra320 and wants that? hmm, ask you for another $12,000 maybe? Maybe its because of his close involvement with the SCSI subsystem in OpenBSD that he's blind anything other then SCSI, he just wants a new toy or whatever, I don't know. But I still stand by my earlier statement that I feel he has not throughly thought this through.
Comments
By marco (but not marco@) (149.169.52.82) on
the 220s is, by itself, a jbod enclosure. it relies on an external raid controller, like the perc 4/dc. they got the "jbod" because they already had the raid part, just like they clearly said
Comments
By Amir Mesry (208.34.41.180) amir.mesry@cadillacjack.com on
By Anonymous Coward (202.45.125.5) on
Yes, this punctuates everything you've said so far. You are looking at this from your limited point of view, while not realising that the picture is much larger and being taken care of by people who do understand the full situation. I am surprised that you have not been jumped on for this. I can only assume most people have stopped reading posts from you.
Manager: We need a version control system put in place for our devs.
IT admin: Okay, we will need fast access storage above all else for that.
Manager: Why?
IT admin: Version control systems tend to cause high I/O (lots of requests) because many files within the system tend to be requested at once, so as to put together any particular version to be given to a developer. Since developers don't always just ask for one file and since we have lots of developers, access time becomes the most important requirement.
Manager: What do you recommend?
IT admin: Disks with highest rotational speed for lowest latency, fastest head movement and top notch queuing and elevator sorting.
Manager: Elevator sorting?
IT admin: If a drive is requested to get a file from the start of the disk, then another from the end, another from the middle, another from the start and then lastly another from the middle again. Instead of the heads moving to the start, end, middle, start and then middle, elevator sorting allows the heads to go start, start, middle, middle, end. This dramatically reduces head movement and thus time to complete access for many pending files.
Manager: Oh, very interesting. So what are the best disks for this?
IT admin: At the moment, the drives with the fastest rotational speed, fastest heads and best queuing and sorting, are 15,000 rpm SCSI drives.
Manager: Oh don't we have scuzzy drives in our mail and database server?
IT admin: Yes, they are also areas that can be very high I/O hogs. SCSI is built for server use and tend to be more reliable also. The fact that the heads don't have to move as much probably also prolongs their life. Interestingly, this storage array could theoretically sustain extraordinarily high sequential transfer rates, but that will rarely, if ever, be needed by this system.
IT staff member: (just walked in, looking at notes on whiteboard) What!? That system alone can sustain over 600 megabytes per second! But you are only connecting it to a 1GHz P3 with just a single U320 controller in an ordinary PCI slot!?!? You see Mr. Thomas (Manager), what have I been telling you all this time? This guy should not be the admin of our infrastructure or trusted to spend any of our budget! The raw throughput this system could achieve, will be severely bottlenecked by the components used to connect to it! Blah blah blah, blah blah...
I think I was a bit to harsh in my reply to marco, sorry for that... but being an excellent software developer does not automatically make them an excellent systems administrator, integrator, or engineer, vice versa.
These guys, who develop a very well respected operating system, know how to spec storage for a core need of the development of that operating system.
I don't want to say that I'm better them him, I don't know him, but It feels like he has an agenda (high-end and costly SCSI system) that is not inline with what the project needs
YOU DON'T KNOW WHAT THE PROJECT NEEDS.
Your right, I don't fully know what the requirements are because I have nothing to do with the project.
So, what you mean to say, is that you are not qualified to decide what best meets the current NEEDS of this project? Then why do we continue to hear so much crap from you?
But from what I do know is the current server is a Pentium 3 with 1GB of RAM and is using 14 18GB Ultra160 SCSI drives in a RAID 5 configuration. Based on the specs I do have I could and will speculate that it is an off the shelf uni-proc P3 and that it has no PCI-X slots, only PCI, and that it's LAN connection is only 100MBit FDX. You can only go as fast as your slowest link and that is the LAN connection, the PCI bus, the memory bus (and how much you have), and lastly then CPU.
Nikolas. Ever thought that raw throughput is not what is needed here? How about access time? What if access time is above all else, the biggest performance killer? The developers know about bottlenecks. They find them in software and develop better ways to reduce them by changing algorithms, for just one example. Not all bottlenecks are clearly defined by product specifications. A 15,000 rpm SCSI hard drive is capable of transfering data at high rates and having very fast seek times due to the fact that when a head is moved into place, it does not have to wait long for the desired sector to come around and move under the R/W head. But maybe, just maybe, transfer rates are not what matters most in the needs of this cvs server. Maybe it is access time that matters most. And when the cvs server is at its busiest, pending commands can be much more efficiently serviced thanks to SCSI. The raw bandwidth that this system is capable of, might merely be an added detail that goes hand-in-hand with this low-latency system, yet never be needed. Who knows!? Me? You? Certainly the developers spec'ing this system know.
I don't fully know what happens in a CVS repo/server but I suspect that it is similar to ftp, http (no dynamic content), nfs, or smb file server and these servers are always network bound and when not networked bound, say for example 1GBit and 10GBit links, they are bound by the PCI bus.
cvs
A Concurrent Versions System server, is capable of allowing changes to files to be recorded in such a way that you could request any version of the file that the system has recorded. To do this with storage efficiency, instead of full files of each version being stored, this system can allow differences to be stored. So when you request a particular version of a file, the original file plus differences may be requested and then combined to present you that version of the file. This has the storage system requesting original files and other files that combine to make up any particular version. When the system is busy, you might find that access time is what matters, not raw brute force transfer rates.
cvs is far different from ftp.
A single 5400RPM IDE drive can saturate a 100MBit link and a 1GBit link can saturate the the PCI bus.
In networking bandwidth, packets per second tend to be more meaningful than bytes per second. You could fully consume a CPU with lots of small packets while getting nowhere near the saturation point of the network interface or system bus. Even though that CPU would normally hardly break a sweat with typical packet size transfers and be able to saturate some other bottleneck. Same goes for high disk I/O demanding applications. If your application causes lots of requests per second, causing head movement, your transfer rates go out the window and reliance is placed more on how quickly those heads can move, how quickly the disk spins (faster spinning disk does not just allow faster transfer rates, but also allows faster access to a sector by reducing the time it takes for that sector to appear under the heads) and how efficiently the system can organise what order those requests should be carried out, to reduce back-and-forth head movement.
The fact that this system might be able to sustain a sequential bandwidth far greater than the network interface and system buses can handle is beside the point. Because that bandwidth might not be desired or ever requested. The point is access times. To get fast access times, it just so happens that you buy a system which also happens to be capable of high sustained sequential bandwidth.
This has to do with SCSI's command queuing and low read/write times do to high RPM drives, 10,000+, with non-sequential / random access patterns SCSI will always beat IDE because IDE doesn't have ether of those features.
That seven times faster SCSI disk, was 7,200 rpm, just like the IDE disk. But yes, SCSI kicks arse for high I/O. A busy cvs server can be a high I/O demand application.
SATA is designed for anything you want it to be. It is only a bus interface, like SCSI, and what really matters, both in SATA and SCSI, is what's connected to it. With drives such as the SATA based WDC Raptor it can be a valid competitor against SCSI, at this point only Ultra160, because the SATA interface supports most of the features that make the SCSI interface the niche product it is today.
What a load of tripe. Every design has strong and weak points. Often when trying to strengthen one area, you end up weakening another. SATA is designed for desktop use, that is its main market. SCSI is designed for server and high end workstation use and that is where it is marketed. Did you notice at storagereview, the SCSI drive running a high I/O benchmark about 65% faster than the SATA 74GB Raptor? SATA may be getting used a lot now in server use, but I have almost always seen SCSI as an option in those same systems, for the higher end.
P.S umm I not sure which SATA drive your talking about... but I'm basing all of my comments off of the new WDC 74GB Raptor, not the 36GB Raptor or any other SATA drive.
Did you look at the storagereview links? I linked to the 74GB Raptor.
SCSI is not dead nor is it going to die anytime soon. It has it's niches, especially in the high-end and big iron markets.
When was the last time you touched "big iron"?
Those people obviously forgot to do random access pattern tests and again this has to do with the SCSI vs IDE bus. IDE will always loose when you compare this aspect.
High I/O, does not mean that the data flows at a high rate, it means that there are a high number of requests (often small). Lots of requests all over the disc cause... random accesses. cvs can be high I/O and I imagine OpenBSD's core cvs server can get pretty busy at certain times of the year.
But with SATA-I, SATA-II, and SATA-III standards you will see in the coming years that it will dominate the entry-level server market and will have a sizable market in mid-level servers. This is do to simple economics and the reasons stated above (and below), you can quote me on that.
I don't doubt that SATA will continue to get better, but so will SCSI. Most importantly though, is that OpenBSD's needs are immediate! Coming years? They need storage NOW!
One thing many seem to be forgetting is that it's the job of a good, one that has has a real I/O controller, cache, etc., RAID controller to play traffic cop to efficiently sort and move data to and from the disks. This lessens the need for command queuing and low access times on the drives if it is done correctly.
Cache is only good if it can be exploited. Cache can also INCREASE access times if the data is not found within the cache. Because the cache is first checked, then it is requested from the platters, placed into the cache and then given to the requesting application. Also, in these situations where cache is of little benefit, the larger the cache the worse the access time due to greater latency caused by searching through a larger cache.
If a drive does not do command queuing, then when the drive is busy it's bus is idle and when it's bus is busy the drive is idle. For an access time reliant server, do you want a disk system that has a duty cycle split between disk and bus that has at all times one of them idle? There is latency getting commands to go from the controller on the motherboard to the controller on the disk. This can be alleviated by having queuing on the drive. Thank God IDE finally has queuing coming with the latest SATA drives. After all these years!
? maybe it was in that link you gave me, sorry didn't read it (yet)... Ultra320/15K vs SATA-I/10K, no contest, SCSI will win all of the tests.
What is the fastest hard disk drive you can buy? Whether by fastest you are referring to transfer rates, access times or both? A 15k rpm SCSI drive. The OpenBSD project just wants what is best for their most important core server.
You can and it does but he stated that he wants to use it in a JBOD array configuration and that is just wrong.
Actually, I don't recall reading Marco state that he was going to use it in JBOD config. And notice that he also mentions the use of a donated RAID card? Why buy something more flash, with built-in RAID when he already has a good RAID card to go with plain SCSI storage?
No I haven't. SATA paired with the right drives can do just about anything Ultra160 can.
That's great. Now how about considering that they are going with U320 drives?
This problem is easy to overcome with other cheaper cost effective solutions, SCSI or not.
SCSI currently provides the best access times. It also provides the best transfer rates, but I don't think that is highest on their priority list. The use of SCSI-IDE bridges is a nasty hack for an important server, which would increase risk of failure.
One of those could be splitting the CVS trees onto separate servers.
Hang on. You have demonstrated and admitted that you know little about cvs and it's requirements, but now you are giving OpenBSD developers, developers by the way, who are currently developing a better OpenCVS system at the source code level, advice on setting a cvs server up?
My bullshit meter is ringing like crazy and thats why I'm "putting up a fight" with his request.
The problem is, that you don't know about cvs or OpenBSD project needs. So you're bullshit meter should be switched off. You should take your fight elsewhere. I think everyone else here is sick of hearing your ill thought out bullshit.
But I still stand by my earlier statement that I feel he has not throughly thought this through.
You are ignorant of the needs in this case. So how could you know whether this has been throughly thought through?
Comments
By mirabile (212.185.103.56) on http://mirbsd.mirsolutions.de/?anargeek
a working copy in the same file, named as the original with ,v
appended.
You know, CVS originally was some shell kludge around RCS.
Comments
By Anonymous Coward (203.166.96.239) on
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
The rest of are debate about disks is irrelevant.
Have a nice day Patrick,
Nikolas Britton
Comments
By Otto Moerbeek (82.197.192.49) otto@drijf.net on http://www.drijf.net
You say you have respect for us, but in reality all you are saying is we're a bunch of hobbyists that like to play with expensive hardware.
You seem to have some special sense to know the current performance and requirements of cvs.openbsd.org. You repeatedly say the people that control and use the machine do not have the insight to be able know what the machine's requirements are and to analyse the performance problems we are seeing. We, who develop the OS the machine is running.
This is all so pathetic I'm begining to wonder why I even post this. I probably should have not. It;s just feeding a troll.
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
I know they can tell the difference, I'm not debating this.
"You say you have respect for us, but in reality all you are saying is we're a bunch of hobbyists that like to play with expensive hardware."
At this point, no I have no respect for the key developers of the OpenBSD project because the more I dig the more I feel I've been lied to. They may need the array but I do not think it was for the reasons they stated.
"You seem to have some special sense to know the current performance and requirements of cvs.openbsd.org. You repeatedly say the people that control and use the machine do not have the insight to be able know what the machine's requirements are and to analyze the performance problems we are seeing. We, who develop the OS the machine is running."
I know they can tell the difference and again, the more I look and analyze it the more disparities I find. At the start I asked marco to justify why this is needed, he never did and this is reason enough for me to question why he needs it.
"This is all so pathetic I'm beginning to wonder why I even post this. I probably should have not. It;s just feeding a troll."
I'm not trolling, I'm just not going to blindly following what they say.
Comments
By Anonymous Coward (202.45.125.5) on
So far, the point you've raised regarding perceived "disparities", amount to maximum sustained transfer rate capabilities versus various bottlenecks. Yet people have stated that high IO matters more, NOT sustained transfer rates.
Fact: High IO absolutely kills transfer rates in physical hard disk systems when heads have to move. While heads are moving, they do not read or write, meaning that maximum transfer rates cannot be realised. The duty cycle between time transfering data and idle time waiting to access data, moves more and more towards the idle time as IO goes up and file sizes go down. If queuing and elevator sorting are not used, then this just becomes ridiculous.
Solution: Whether IDE or SCSI, high IO is going to mean that maximum transfer rates will not be attainable. So the best solution is to choose the disk system that has the best high IO performance. SCSI is DESIGNED for these scenarios, because that is what is typical of the demand on servers. IDE was NOT designed for this, because it is an end user technology. Newer SATA drives now support TCQ due to heavier loads placed on drives due to modern multitasking operating systems, but these SATA drives are still not designed to meet the high IO demands that SCSI is designed for and besides that they only go to 10,000 rpm and their seek times are slower.
So accounting for the best performance and security, the fastest SCSI drives and enclosure that the project can afford, along with a good RAID card is the best solution for the money. This is pretty basic stuff.
So as to bring this to a quick close, could you please summarize all disparities you have found, in point form? Please provide for each:
* The disparity
* Evidence of it
* Why it is wrong
* Your solution
At the end of the day, OpenBSD is developed to an extraordinarily high quality for next to nothing, by selfless individuals, many of which do it on their own time and dime. If their time can be more efficiently served with the fastest array on the planet, and the community of developers and users are willing to pay for that, then fantastic. Even if they were just wanting a neat techie toy to play with, I would not mind throwing money towards it because they deserve a lot, given the work they put into OpenBSD. Hell, I'd be happy to buy pizza and beer for these fine folks.
Lighten up will you?
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us/
The reason for a RAID array are plentiful and all most of them have been recorded throughout the posts. If you cared enough to actually do some research you would have found numerous posts of key OpenBSD folks that talk about cvs.openbsd.org and what it does. Let me recap some of the reasons:
* Practicality
* Known stability
* Service contract
* Future growth
* Redundancy
* High capacity * Environment monitoring
* Everything hotplug
* High IOP throughput
* High performance
I don't know who you are but I do know what you are. Besides calling me a liar and cheater I have seen nothing constructive coming out of your fingers. Frankly I am amazed on how you question my integrity. Have you ever thought of the possibility that just maybe there are other reasons for buying this type of equipment? Has it ever crossed your mind that there might be some more OpenBSD folks that have experience with these machines? Could it be therefore that it is a really practical solution for us? Could it also be that Marco did not come up with this by himself but had the necessary people work with him to come up with this? Maybe even Theo? Could it even be that the current similar U160 setup is no longer cutting it and it simply is a natural upgrade path? No, of course you haven't because you are too busy "standing up" and fighting the "struggle".
Based on your postings it is also apparent that you are a computer enthusiast who likes to goop together some hardware. I find that amusing too but not when my most critical data is at stake. The fact that you have never worked with high-end hardware does not warrant your insults. For those of us who have worked in the largest of data-centers this is a practical and, believe it or not, economical solution.
And before you go off again and tell the world that Marco needs a toy let me make sure you are aware that I have racks full of these things to play with at work. I also have racks full of big iron servers to play with. Oh and I also have FC equipment available. My sandbox is pretty cool. I will not see anything in return for this besides a reliable cvs machine and insults from people like yourself.
I'll waste one more paragraph on this. I tested several SATA RAID controllers and found out that they are not reliable. Besides reliability issues I also noticed that most SATA RAID controllers are slower than a single SATA disk. But oh why? Please tell us more. Well, maybe that has to do with the fact that SATA does not queue IOs and therefore a RAID array is as slow as it slowest disk. But what about this new TCQ thing? Well, that one is not implemented in any SATA RAID card because there is no clear spec on it; it is all vendor specific hardware that require vendor specific drivers. On RAID cards we like to be able to mix vendors since it is not practical and feasible to have the same type of drives.
I can go on for a while but I have wasted enough time on your insults.
By Anonymous Coward (203.166.96.236) on
By tedu (64.173.147.27) on
duh, no it does not. actually, it clearly states, "-a server with 32M of memory or even less can handle a fairly large source tree with a fair amount of activity." then goes on to mention you need "slightly more than the size of the sources in a single directory, or two megabytes, whichever is larger." for a checkout, or for diff "For example, if you want to check in a file which is 10 megabytes, you should have 100 megabytes of memory on the machine doing the checkin" 10 megabyte files are pretty far and few between in the openbsd repo, and they don't exactly get checked in frequently. 1GB is plenty. notably, it'd be pretty obvious if the machine were swapping.
that entire section fails to mention any consideration for disk setup. in fact, based on that section only, one could conclude that placing the cvs repo on a tape drive would be a reasonable setup.
but all this coming from a guy who thinks a cvs server is no different than an ftp server. "from my perspective, as a downloader, it works no different then an glorified FTP server." here's a clue. from my perspective, as a developer, it works very much differently from an ftp server. hmm, what does an ftp server do? it serves up large files, sequentially. what does a cvs server do? it *modifies* many small files. that's read, modify, write.
besides the fact that this is also the machine that's the main nfs server. you know what a big performance killer with nfs is? access time.
By Amir Mesry (208.34.41.180) on
Comments
By Nikolas Britton (12.223.129.46) freebsd@nbritton.org on
Didn't understand what he meant by "JBOD with U320 disks", my bad.
By Anthony R (68.145.103.21) on
By Anonymous Coward (67.78.160.141) on
By Wes Williams (68.213.136.249) on
There are likely more of us that agree with the use of SCSI for this project but just aren't as vocal [whiney] as those seeming to use this board as a tear duct.
Comments
By Anonymous Coward (24.139.14.28) on
By rainer_d (62.158.173.45) rainer@ultra-secure.de on
The array is going to serve small files, randomly picked (mostly, I presume, from comments), so anybody who has done some research other than comparing prices and capacity can do his own math about how a SATA-solution might or might not perform.
Yes, you can cobble together a 4 TB "file-server" for less than 5 grand today - but that's not the purpose of this machine - and from my experience, these "solutions" can be a real PITA to maintain.
By Blake (81.57.19.80) on two one one two dot net
By Marco Peereboom (67.64.89.177) slash@peereboom.us on http://www.peereboom.us/
For you complainers out there you might want to learn from these folks who selflessly donated for a cause they deemed important. Here is the "got it" email: From: slash@peereboom.us
Subject: cvs.openbsd.org upgrade, we're there!!
Date: March 15, 2005 6:31:49 PM CST
To: misc@openbsd.org
We received over $12500 USD for the purchase of a brand new RAID array for cvs.openbsd.org! I want to thank everyone who stepped up and donated some of their hard earned cash. I collected names from several folks in OpenBSD who manage various accounts so if I missed you please let me know and I'll correct it. I also got some emails from folks who sent money orders and have not been able to collect on them yet.
We really, really appreciate all the donations!!
Here is the list of names in no particular order of the folks who donated:
Blake Willis Christian Jones Sam Nicholson Mike Tancsa Matthieu Herrb Nick Humphrey Michael Shalayeff Andres Martinez Olivier Matthey Astar Computer Consulting Daniel Zinngrabe Sean Hafeez Alex Valentine Raymond DeJean Donald Sinclair Jason Napoli Daniel Trombley Wes Williams Andrew David Anthony Roberts Global Freelance Brandon Davies Adrian Basescu J Scott Deacon Robert Copsey Ronald Nowling Yan Watkins John R Danks Stephan Tesch Dominic Crowson John Katagawa Brian Falk Ido Admon Matthew J Rowley Garance A Drosehn Georg Wendenburg Loic Tortay Rob Bricheno Steven Fettig Anywhere Technology Corp. Mike Miller Mark Patruck Patrick Bergamin Kevin R. Smith Eric Moore Teoh Cy Boon Dean Schuetze Sam Smith Francois Chambaud Mark Hesselink Scott Ellis Jorg Willekens Matthew Weaver Sean Underwood John Breese Derek Morr Jason Meltzer Noah Aklilu Gregory Zentkovich Stefan Wollny SMAT Engineering LLC Miklos Koevari Kirk Strauser Damian Gerow Matthias Herrmann Philip Dicke Anthony Ten Broeck Hugo Lombard Adam Getchell Keith Scott Chris Ferebee Keith Scott Gregory Steuck Jon Gordon Sean Richardson Benedikt Heinen Jason Santos Lars-Erik Lemonis-Nicolas David Steinberg Benedikt Heinen Karsten McMinn Evan Seitz Mark Peoples Peter Ibsen Hansen Ian McWilliam Nico Meijer Rodolfo Gouveia Alf Schlichting Peter Ibsen Hansen Ian McWilliam Nico Meijer Lars Cleary Rodolfo Gouveia micro systems Xavier Mertens Chris Murphy Stone Networking Ronald Smith Jonah Benton Greg Girczyc David Cathcart Fredric Cohen The Violin Case Jan Joris Vereijken Masoud Sharbiani Wijnand Wiersma Pascal Lalonde Cristian Bratu Masoud Sharbiani Dan Price Ricardo R. Aliwalas Kurt Miller Erik Tigerholm Okan Demirmen Nicholas Marriott Jan Joris Vereijken Johan Torin Otto Moerbeek Sean Ellis John Pugh Wolfgang Anger Benjamin Crowell rob casson Emilio Perea Kenneth Stox Garhan Attebury James Bishop Sean Ellis Howard Owen Ferret Ferret James Raden Otto Moerbeek Dave Shanker Bill Saunders Robert Thrush Thorsten Glaser Marco Feenstra Jean-Gerard Pailloncy Jonathan Hart Weil Tristan Mike Gruber Abraham Al-Saleh Marco Feenstra Chad Remesch Mykhaylo Shalayev Craig Hammond James Herbert Sjoert Bakker Jelle Posthuma Isak Lyberth Oliver Morais Michael Erdely Vikram Kulkarni joseph r franklin Ross Lonstein Stuart Henderson Eddy Carroll Simon Slater Jeff Martin Carson Chittom Jelle Posthuma Sjoert Bakker James Herbert Gernot Poerner Christian R¸ger Oliver Morais Craig Hammond Michael Erdely Jamie Eubank Poetic Pollution Dimitri Georganas Vikram Kulkarni joseph r franklin Doug Clements Chris Bensend Brandon J Creighton peter varga Ross Lonstein Johan Torin Stuart Henderson Eddy Carroll Simon Slater Marco Peereboom
Comments
By Anthony Roberts (68.145.103.21) on
I've no doubt that everything that makes working on OpenBSD less annoying will result in better software for everyone. I've worked with overloaded CVS servers, and "annoying" is probably the most polite of the words that comes to mind.
By Anonymous Coward (71.0.126.14) on
"For you complainers out there you might want to learn from these folks who selflessly donated for a cause they deemed important."
The only thing to be learned here is that you only have respect those that kiss your ass and tell you what you want to hear. And that many in the OpenBSD community become the very trolls they accuse others of being when someone calls you out for it. Be proud, you could have made much more effective use of those funds and you know it.
Comments
By Anonymous Coward (202.45.125.5) on
You keep going on and on about raw throughput and associated bottlenecks and yet keep side-stepping the access time issue that people are bringing up.
You don't seem to want to address the thought that raw sustained throughput might not actually be near the top of the priorities for this need. Have you ever seen the overall transfer rates for lots of randomly accessed data? With typical tests, my IDE drives typically drop by about 50 times when comparing sequential transfer rates to random transfer rates.
Yet in light of all of this (your lack of understanding and valid points being raised), you continue to claim that Marco has an agenda, has not thought this through and that users willing to contribute to this fine OS are lemmings.
You know you don't have to be such a waste of time. You can choose to do things that you can be proud of.
Comments
By Anonymous Coward (71.0.126.14) on
You seem to have me confused with someone else who had similar concerns. There are a few of us that have posted them here, which is vindication that I am not alone as seeing this expense as ungodly. Stop trying to make us the same individual. Though is funny to watch all of the modding down based on the who, rather than the what has been said.
I don't think Marco has an "agenda". I think he's not being as fiscally responsible as he should be and throwing contributors money away on bloated equiptment costs. I am not in favor of any specific solution, I am only in favor of something more economical. It seems to me that the FTP server and the AnonCVS servers would need the highest performance, while the main CVS repository, serving only developers and mirrors, would need the most secure redundancy.
I also feel Marco has been rude to and about anyone who has questioned him on this, along with several other posters here. The very fact Marco sought to chastise us and call us "complainers" in his thank you to the contributors, to hold us up as needing to learn something from them, shows the very level of arrogance and godhead that forced me to second guess his judgement to begin with. It also shows a blatant level of wounded egoism to go out of his way just to mention us.
It has forced me to consider that from my perspective, and probably a few other individuals reading all of this, that contributing to OpenBSD is probably best done only through CD sales and direct hardware donataions. Or at least directly through Theo, who I trust to better dictate selflessly the real necessities of OpenBSD, seperate from the ideal equiptment he would like to see, i.e. "Over here is what we need. But over here is what we would like to get."
Comments
By Anonymous Coward (24.139.14.114) on
Please read. Theo knows what is going on and the PowerVault is replacing a
similarly spec'd machine.
By Chris (24.76.170.207) on
It is lucky for me and everyone else using OpenBSD that the developers don't listen to people like you and just keep doing what they think is best. But hey, it's only worked for about 10 years, so maybe they don't know what they're doing.
There's a difference between concientious objection and just being difficult.
By CyBoon (192.245.59.130) on
To disprove my statement, the party at 71.0.126.14 need only furnish
answers to the following qestions (And a simple guide to what is
required from his/her answers is given as a convenience):
1. Justify the statement:
"It seems to me that the FTP server and the AnonCVS servers
would need the highest performance, while the main CVS
repository, serving only developers and mirrors, would need
the most secure redundancy."
A good answer will contain:
- definitions of 'performance' and 'secure redundancy' in terms
of measureable data points for the services so mentioned.
- Data collected at a site (or set of sites) comparable to the
OpenBSD site for the metrics defined above and a description
of the protocol used for collecting the data.
- Justification of the statement made with reference to the
data so presented.
2. Justify the statement:
" I think he's not being as fiscally responsible as he should
be and throwing contributors money away on bloated equiptment
costs."
A good answer will include:
- A proposed alternative spec, including price quotes from a
reputable supplier, which clearly demonstrates a cost savings
of at least 5% as compared to the one proposed by marco.
- A set of contingency plans to be used over a reasonably
estimated lifespan for the proposed alternative, including
cost and time estimates for when something goes wrong with
the proposed alternative spec.
- Justification of the statements made by demonstrating that
the proposed alternative is more cost effective to acquire,
deploy and maintain.
These are simple questions that trolls simply won't answer.
By Blake (81.57.19.80) on two one one two dot net