Slashdot Mirror
Should a Web Startup Go Straight To the Cloud?
Javaman59 writes "I am a one person company developing a web site from home. The site is hoped to attract millions of accounts and daily hits (just to give an idea of the scale of things, as its important to the question). My infrastructure is currently Visual Studio 2010 on a PC. To progress the site I need to set up version control, continuous integration, and staging. I have a Win2008 server VM, with all the Windows software (free and legal) to do this. However, I am only just competent as a Win admin, and I foresee each step of the way (setting up a domain; SQL-Server, etc) as a slow, risky process, and a big disruption to development. Should I forget my VM server (it will make a nice games machine!) and just go straight to the cloud for all my infrastructure?"
You're not going anywhere running windows unless you had shit loads of cash behind you.
I'm sorry, but if you need to ask Slashdot on something like this than your not qualified to do what you want to do. Nothing personal but your only going to have hours to days before your website is hosting malware or gets turned into a spam relay.
Don't use MS products if you want to scale.
Google App Engine. Steep learning curve but worth it.
in house has less lag and more bandwidth then working over the web from your home. How good is your upload?
It's porn isn't it?
The reality is that a "just competent" person is far from qualified to make this choice. It is easy to think that "the cloud" is an easy answer to scaling problems, but if you do not design your service/software with horizontal scaling in mind, you may find that your service does not scale up on "the cloud" any more effectively than it could on your own servers. "Cloud" does not imply scalability, it implies out-sourced infrastructure which is accounted for as an OpEx, rather than a CapEx. You still must plan for scaling up.
Seriously. What is this on Slashdot for?
Disagree != mod troll.
A bit of alpha testing with a few friends from your home box is definitely OK. But if your app is going to be as popular as you think it is, get it running where you can scale it up fast. Or you may end up pissing off your customers with poor performance and they'll just leave.
Getting your app configured properly for 'The Cloud' is going to be a prerequisite for the production version. Work those bugs out now rather than having to patch the 1.0 version.
Have gnu, will travel.
Why would you want to host your website on an MS OS anyway, let alone one you don't even know how to administer properly.
My advice, look around for a good hosting deal (with good backups etc.) on shared hosting. Buy a hosting plan that can easily be upgraded to something dedicated as and when (and if) needed. Use your machine as a developer machine with all that entails (backups, version control etc.).
Forget about "the cloud". You're dreaming at this point. "The cloud" is something for when you actually need to radically increase the number of visitors that the site can handle.
Appended to the end of comments you post. The maximum is 120 characters.
Yes, use the cloud if you expect a lot of growth. No, do not use Microsoft products, they are great for the enterprise, but not for a one man shop. Use some nosql tech, Memcache, and some scripting language with your favorite JavaScript library.
Unless this is some kind of troll, I'm unclear why you would have picked a platform like Win2008 for a large-scale web server, when a LAMP architecture is easier to manage and more easily portable to the cloud if you do decide to go that way.
Uh, he specifically wants to spend more time on developing and not worrying about the hosting part. He also understands the risks in it. In that regard using cloud hosting and storage could be really good, as long as you calculate the price comparatively.
Since he is using Visual Studio 2010 for development, I even suggest using Microsoft's Azure. It integrates beautifully with VS and you can run your code directly in the cloud, while debugging too.
I really don't understand why a small web startup would go with Microsoft. The licensing costs when you (hopefully) start to scale up are going to kill you. There's a reason that all the big-hit startups over the past decade weren't standardizing on Windows as their web platform.
Astroturfing
You may be right - Look at all that MS name dropping, and this guy is running this out of his home? He must be one rich guy to pay all the licensing fees, especially if he expects to scale to "millions of accounts and daily hits". Either itâ(TM)s pure Astroturf, or this guy just isnâ(TM)t living in reality.
If you want news from today, you have to come back tomorrow.
What you really need to do is find an entity who can help you with the tech mechanics. That entity could be a friend you promise to reward later, a business partner you legally go 50/50 with, an independent consultant you hire, a company (large or small) you hire. But you're asking really basic questions about stuff, so you obviously need some help. Moving to the cloud just moves the problem to some place you can't touch; it doesn't address it.
(If you're offended by the suggestion that you need help, you need to adjust your thinking significantly, or abandon the idea of going into business for yourself. No one person can do everything, and any successful business person will need to realize that early on.)
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
I call shenanigans. The username of this "Windows developer" is "javaman". He names the major Windows dev and SQL Server brands but doesn't mention Azure. He refers to "continuous integration" but pretends that setting up SQL Server is hard. Methinks this is a plant...though I'm not sure for what.
how web site millions daily hits?
His name is javaman and he's using visual studio 2010.
Do you even lift?
These aren't the 'roids you're looking for.
This being Slashdot, one expects anti-Microsoft bashing regardless of subject or context, but... there's a reason Apache has 60% of the market to Microsoft's 20% (citation: http://news.netcraft.com/archives/2011/05/02/may-2011-web-server-survey.html), and it's because the LAMP stack is vastly superior to Microsoft's for running public websites. For an internal corporate infrastructure, you can at least debate whether AD and SharePoint and Exchange/Outlook and so forth are worth using, but on the public web, whether doing simple hosting or web applications, you need a *very* compelling reason to want to go with Microsoft.
are useless little children. You're better off finding a few books and professional sites to set you on a proper path.
'The cloud' does not set up your infrastructure. It does not design or enforce your version control. It does not harden against SQL injection attacks.
Your 'infrastructure', Visual Studio on a PC, is not infrastructure. That is merely a basic dev box.
Staging? Needs to be a small scale duplicate of production.
Do you have any clue? Apparently not. If you did, you'd hire someone else that actually knows how to do this. Because you clearly do not.. You have an 'idea'. An idea that you apparently cannot bring to the public.
Sorry, but that the truth.
Yes. Don't take on anything more than your core skills, or the other stuff will eat into your time and stop you doing what you aim to develop. When starting up you might even find free cloud hosting whilst you develop.
You will want to move from Microsoft products as a one-man band, as this will make your cloud choices cheaper and more varied. Look into PostgreSQL or MySQL, and PHP, Java, Python or any one of the myriad other Linux web server languages.
But do put some thought into how you are going to scale your platform so that it will run on the dirt cheap hosting platform initially, yet expand and scale across multiple cloud hosts down the line.
You should totally be looking to do FORTRAN on z/VM... It's the only way you will be able to handle the "millions" of accounts you will be creating to pump your own image.
--WooooHoooo--
yes.
more info:
forget windows.
learn about cloud or find somebody who can help.
use the many web services available cheap for SCM, deployment, task management, etc.
if you only want to focus on development for yourself, then again you need to find someone to handle all the other stuff.
"Java" could refer to Coffee indicating a complete lack of knowledge about software which aligns well with the post
So what? He has UID 524434 so he has been around on slashdot for a quite while. That just shows he has past coding experience in Java, and quick google query shows he is coding with C# now. Java->C# is a natural progress (as the languages are similar, but C# is better) and Visual Studio 2010 and Windows environment makes a lot of sense for C#.
starbucks?
I'd say it's negative advertisement: this guy got Visual Studio 2010 and a Win2008 server VM loaded with Windows software, and he cannot achieve what he wants.
1. Get used to the idea that your development and production environments are two distinct animals to be kept apart on purpose. Using one machine to develop and to host the production service is a recipe for disaster.
2. Stop using the cloud as a magical entity that holds everything that is online. You can go to many web hosts and pick up a decent IIS and SQL Server hosting plan for just a few bucks per month. If you build your solution so it can allow multiple server instances, you can always move it to the Azure Service or Amazon's cloud service.
3. "Hoped to attract" is not going to cut it. You need to have market research and a lot more before you can even talk numbers, otherwise you are down to a "cool idea that I hope catches on."
4. Offload your non-programming duties on somebody else. After many years programming, I have found that I was consistently miserable when my job description included responsibility for the infrastructure. It doesn't mean you stop learning about the infrastructure, it is just that you just concentrate on what you are building and somebody else is stuck keeping the servers alive.
Pedro
----
The Insomniac Coder
The guy above said you're not qualified, but the young and inexperienced have come up with plenty of new and innovative stuff, so:
I think the question is: If not cloud, what?
In no case would I run the site from home. You'll probably get your home Internet yanked.
Cloud usually implies the ability to instantly increase/decrease the size of your server. I don't think you need that at the start.
You could go with a cheap VPS. In fact, I think that's what you should do. You should be able to take a stock Windows VPS and install your application, and have everything working. Either write an installation how-to, or reduce the steps to a script (PowerShell or whatever).
After you're able to do that, you could start looking into cloud provisioning, separate database server, database replication, DNS proxying, round-robin DNS, backups/rsync, https and SSL certificates.
You'll need to run email on the server, too, if only to send notification messages to your users (or to yourself). So you need to learn about how to administer and email server. Or rent and Exchange Server.
You also need to learn about CANSPAM requirements.
You'll need to have some kind of monitoring service to alert you to problems with your server. collectd is great for this on Linux.
You also need to look into which ad service you'll be using. Or alternatively, which payment service. Don't keep credit card numbers on your site, don't manage subscriptions by yourself. Let the payment processor do it.
I'm not a lawyer, but I play one on the Internet. Blog
Since scalability is a concern for your app, you probably want to architect your server around a LAMP stack. For version control use git or svn. Windows licenses get more expensive as they accept more users, whereas LAMP stacks have only the cost of maintenance.
"Going to the cloud" vs "hosting your server" is only a question of which hardware will host your machine. It's irrelevant to you right now. If you're pinching pennies it's almost always cheaper and faster to run LAMP apps on local hardware and then scale to remote hosting when you have enough users to warrant.
The nature of your question indicates that you don't know all the questions yet. I think you need a few more months of R&D before you call yourself a web startup.
...but you're in way over your head. Or at least, you will be if you get even a fraction of the hits you are planing on.
Some simple advise:
1. Start small, and grow slowly. Don't start off with the cloud. That being said, look to see what services the cloud has to offer, and run similar services on your desktop. You'll notice that a lot of the clouds offer LAMP with ssh access (or something similar). Do the same at home.
2. Go with a free software stack, so you can afford the mistakes. (LAMP, or something BSD or such.)
3. Learn the tools of administration. That means eating your dogfood: If your platform is going to be LAMP, make all your machines run Linux, so you know how to deal with the simple administration stuff along the way.
Help! I'm a slashdot refugee.
The answer is not all that simple and straight-forward. First you have to decide what level of "cloud" it is you want? Is it platform-as-a-service (PAAS) or infrastructure-as-a-service (IAAS)? Your situation will be very different depending on what you choose. Assuming you are interested in going all out cloud and insist on going for microsoft products, you should read up on Azure and then decide if the gains are worth it. You should especially read up on the pricing model and see if it fits with your model for generating income.
I think you answered all your questions. It's probably cheaper and easier to use a solution that already exists, so as long as your technology and business plan doesn't rely on having your own physical servers, then go "to the cloud!"
I assume you have either DSL or Cable for your high speed Internet connectivity? It doesn't matter if you have their top tier, it's not going to be anywhere near enough to handle the demand you are expecting! You need a hosting service, like 123ehost.com or 1and1.com, etc. Their prices are very reasonable, and they have the bandwidth to handle large demand.
However this is the plan and the reasoning.
- Windows Server 2008 R2: - ASP.Net + MVC3 + Entity Framework 4.1 w/ POCO + C# is simply a fantastic development platform. You are a single person which likely means your time is the optimizing factor, NOT cost. Otherwise use Linux + Apache as it will save you in the long term.
- MySQL over MSSQL: MySQL is a nearly identical engineering experience for a site of your scale and will save you 10s of thousands of licensing costs in the long run. See the MySQL .Net connector. If you are bold try MongoDB, however it sounds like you are most comfortable with RDBMS and we are optimizing for your time.
- Source control: GIT, then SVN, then Perforce; by order of preference. The first two are free, the last one is used widely in professional shops (MSFT for example). Your choice.
- Cloud: Assuming you have the dev chops to target it, don't. As a one man operation, doing this now will sap your time and you will never ship. Design for the scale you have and not the scale you want.
- Licensing: Google Microsoft BizSpark and enroll. As a startup you get something like the first 3 years unlimited licenses for a few hundred bucks.
Notes:
1. This optimizes for getting you to market quickest.
2. This minimizes your long term licensing costs (Windows Server ONLY).
3. This assumes that once you get to market and start acquiring customers, you will get traction needed to invest in fancy things like building for the cloud.
Everything else is premature optimization.
If I were you as a one-man operation, I would definitely look into a cloud-based service - not so much for technical reasons but for reasons of practicality. It is challenging enough to manage your sales, marketing, and accounting let alone your information technology. The cloud will at least alleviate one significant headache allowing you to focus on earning money. I run my own IT consulting business part-time and I use Google Apps for Business because the last thing I want to do is maintain an Windows Domain Controller and Exchange Server. By all rights, I have experience as both a Windows and UNIX admin but my focus needs to be on drumming up business, not worrying about malware, viruses, and database corruption. I believe you can find competitive Windows Server Hosting from companies like 1and1 at http://www.1and1.com./
Absolutely! A datacenter keeps everything running under optimum conditions: redundant power grid connections, redundant backbone connections, air conditioned environment, restricted high-security access, and professional systems/network/administrative engineers. This option also allows you to purchase only what you need - and you can easily upgrade as those needs grow. Customer support is usually available. And if you have specialized needs, you can rent a server (virtual private server or physical server), or you can co-locate your own server in a datacenter. It does cost money, but think of how much more valuable your time is, compared to the time it takes to administer your server. Also consider the potential loss of income due to downtime. Having run a hosting company in years past, I can tell you the business is cut-throat, with other companies racing to be the cheapest. Prices have been driven so low that it will cost you more for a cuppa' joe at Starbux than it will for a month of hosting. I think the business decision is clear: prices are cheap enough to just leave it to the pros
You say "millions of ... daily hits". For simplicity, let's say that you get about 1 million hits per day; that's about 10 hits per second, and that's if they're spread out evenly throughout the day. If it's fairly business-centric and USA-centric, then let's say that you get about 90% of those hits during a period of about 10 hours; that's more like 25 hits per second. Now how long will it take your server, on average, to process one hit (taking multiple processes/threads/etc. into account)? The difference between 0.02 seconds and 0.12 seconds now determines whether it gets swamped or not.
If you do run this type of volume on your own kit, then you'll need to pay serious attention to (1) optimizing for processing speed (including volume of data sent back and forth between your site and the user) and (2) using multiple web servers and/or database servers with load balancing.
You are like someone trying to start a restaurant who is asking the web for advice on the menu when you should be focusing how good the location is and the rent. Your lack of business acumen (even compared to your technical expertise!) is what is going to kill you. Is your idea anything more compelling than getting hits on pages decorated with adsense crap? On the evidence you're going to get fucked either way. LAMP is probably the better bet but if you go into the cloud you'll be paying for those clicks: dearly if only a small percentage are revenue generating. MSFT is not necessarily inferior technically, but it's targeted at Enterprises with deep pockets, not 1-person startups: you will go broke with the up-front costs.
None of the comments so far have any value to them. And all of them are rather biased towards LAMP architecture.
I am in a very similar situation. The reason for me choosing MS based stack is because I've spent the last 10 years working with .NET and C# (one very large scale Enterprise projects) and you just cannot throw away experience like this, and, realistically, you can not use mono (or I personally would not) in a live environment. Secondly, if you compare the actual cost of hosting something in the cloud (say Amazon EC2) the premium cost for using Windows is comparatively small.
My personal plan - is to go straight into the cloud (either Amazon or Azure). I really don't want to spend much time or money on setting up my own hosing environment, let the people who are supposed to be good do it for you!
A couple of items jump out at me...
- "a one person company developing a web site from home" that "is hoped to attract millions of accounts and daily hits..."
- "I am only just competent as a Win admin..."
No way this is a real question !
M$ shill! Ass kissing away in Redmond?
Depends.
I would use something like cloud services to minimize startup costs if it made sense.
I would not use the cloud though if:
1) Your startup requires security. Cloud services you have to remember own your data. If they decide to give it out with or without your permission, you have no control over the outcome. In fact, you may not even have any legal recourse either.
2) Lots of storage. It is not clear what the track record is on Cloud Storage. I may be cheaper for sure, but it may not be very reliable. Further if your project or startup is going to need lots of storage, it might not be practical to consider cloud storage.
-Hack
Got Geometrodynamics? Awe, too hard to figure out? Too bad.
If you're already developing with Visual Studio 2010, and using MS SQL as the backend, why not look at Microsoft's Azure platform? It integrates with both, and your web application should take less to run. Plus, I saw some items that they had promotions for people who get their apps validated (marketing funds and Office 2010), and something about free or discounted trials on Azure.
"Uh, he specifically wants to spend more time on developing and not worrying about the hosting part."
Considering the problems cloud services have had this past year, he's sorely mistaken if he thinks he'll be spending more time developing.
The cloud was not even viable for our small business - not reliable enough, and the people running the cloud are just about as clueless.
Watching Reddit go down hardcore because of cloud failures was even more of an eye-opener.
If you can't be responsible for every part of your business, you don't need to be in business, PERIOD.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
MS Cloud solutions is a lot more expensive then linux based cloud solutions or even GAE or Amazon. Poor choice to do a startup with MS toolset. If your doing a startup you need to keep your fixed costs as low as possible. MS Licensing plus hosting is not top on the list for startups
While everyone is pissing on your parade I would encourage you to go for it, because even in failure there us knowledge that can expand your skill set and if you become proficient but not profitable you can at least have a list of skills and lessons learned for future interviews. It shows you can and are willing to learn on your own and not afraid to go for it.
I would recommend perforce as a source code management system, its free for 2 or fewer users, and 5 workspaces.
I would look at square space as a host
First off I agree with one post, if you have to ask slashdot, you probably are in over your head. There are many $5-$10 hosting sites that will give you all you need, a directory to drop your asp.net files in and a shared database. If your site grows (99.9% of the time it won't), then swtich to a virtual or dedicated machine hosted by one of those large hosting companies. And for all you MS bashers, MS has many price points, from free to enterprise. With open source many times you still have to pay just as much if you get big. And if you have so much traffic that you have to buy all enterprise level MS products, it shouldn't be that much of a cost cus you should be making a lot of money.
I'd recommend you drop Visual Studio and Windows and "go to the cloud" on an environment that is already scaled out. There are a few options, but I think one of the easiest to set up and get going on is Google's App Engine system. I don't know what your preferred programming language is, and if it's not Java or Python (or Go), then you're going to have to switch, but all of those are easy languages to learn and the time required to learn them will be trivial compared to the rest of what it's going to take you to build a significant site. The App Engine SDKs are pretty easy to work with and provide a lot of powerful tools, and your site will be running on Google's infrastructure so you know it'll scale as far as you need it to. The free quota will allow you approximately 5 million pageviews per month, so there's plenty of room for initial growth. When you get to where you need more than that you should also have some cash flowing in to allow you to buy more quota.
If you're concerned about being tied to Google (a valid concern), I'd also recommend that you put some thought into placing a layer between your business logic and Google's APIs. I wouldn't make a huge investment in that, because it's the sort of thing that can soak up a LOT of time, so much that you never actually get your site off the ground, but a little thought up front will make it much easier to migrate to your own platform when you have the revenue to justify hiring all the people you need to do that (because it's a BIG job).
The nice thing is that you can start small, for free (other than your time, of course), and have plenty of room to test your ideas and your approach on the small-to-medium scale before it actually costs you anything, other than your time. Then by the time you're ready to scale up, you should know what you need, and hopefully have the cash to fund it. Or, if it doesn't work out, at least you minimized your sunk cost.
(Disclaimer: I'm a Google employee. That actually has little to do with my recommendation, other than that my employment has motivated me to play with App Engine and I've been impressed with what I've seen, but I feel it should be mentioned.)
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Hi Javaman59. I run my own web startup and in the first year found myself moving from a dedicated windows server to a cloud based linux solution for the significant performance improvements. The learning curve for linux administration wasn't too big and if you're already developing for .net/mssql I'm sure you can handle it. PM me if you want any specific recommendations or articles to start with. Good luck, it sounds like a fun project.
P.S. 2nd vote for Azure if you're sticking with a M$ platform.
LOL, so you've never called in an outside contractor for ANYTHING? You guys cleared the land, built the building, manufactured the microchips, etc etc etc. Replace "cloud" with "low startup cost data center" and rethink your statement please.
A cloud system is a good thing when you have a volatile usage scenario and you have the means of adapting to that automatically. If you start a new service you do not need such things. If your assumption is that the business will increase steadily you better real hardware. The present cloud services are too expensive for normal predictable loads.
He would fail if he didn't ask for help.
Grammar nazis are to this community what excrements are to gold.
It is going to cost you more and be much more of a pain in the ass to host your own physical WIndows server than it will to get a virtual server in the "cloud," so yeah, this isn't really optional. Trust me, you don't want to have to worry about babysitting your server in a collocation facility. Plus, it simply will not scale. Before you get even 1/2 of the way to the point of millions of users, you'll have a staff and at least one full time admin,w/ dedicated database server, applicaiton servers, load balancers, etc.
Though it sounds like you're getting way ahead of yourself. Just worry about getting your site out there for now. Get a virtual server.
he site is hoped to attract millions of accounts and daily hits (just to give an idea of the scale of things, as its important to the question).
Dude, that's sooooo 1999! So, how are you going to drive revenues? Advertising?! *condescending snicker*
Do you have contacts in the VC world? Know some dot-commer billionaire to give you some tenuous credibility? No?
You'd have a better chance buying a lottery ticket.
Use your talent and skills to develop something fringe (i.e. new and unique.) instead - I seriously doubt you're doing anything that hasn't been done to death.
Have you considered changing your infrastructure from Visual Studio on a PC to gvim on a Linux box? Also you might benefit from using apache2 and mysql as your text editor. You are quite right about setting up a domain, it is a slow process fraught with peril. Normally I'd advise hiring a developer to do this for you, but as you can't be too careful, make sure the geek you hire has expert level skills in medieval sword techniques. Since in several US States and some countries asking about ancient weapon skills in an interview is illegal, you can effectively screen for this by asking candidates if they prefer Harry Potter, Enders Game, or Twilight. PS If you are not already living in the US, considering you are a sentient one person company, you might consider relocating here. We've become the first nation to truly recognize the rights of corporations as people, and you would not be treated like a second class citizen (taxed, held liable for your mistakes). If your website takes off you could even wield massive influence over our electoral process, and how fun does that sound?
All those comments about using LAMP may be true, but when you're comfortable with Windows and not with Linux, stick with Windows. When the application you're going to run *can* run on Linux, then you may want to keep it that way, so you can move to Linux later on. When you're making money you could hire a Linux admin and then move. It may be cheaper in the end.
What is "the cloud" for you? I would think of Azure or Amazon EC2. I don't know of the fees that Azure has, but I believe Amazon has reasonable small servers (micro instances) that can be "scaled" (moved to larger instances) quite easily. At work we use Amazon, and I think it is great with the instant backup snapshots and all options that you have. But do you need it, and do you need it now?
My advice is this. Stay on your server at home, and use that as long as possible, unless you know that it's more expensive because of power and internet and license costs. In the mean time, start up an instance at Amazon. Configure it, get used to it. Run it, test your application on it. Set it up like another staging environment. Then shut it down, and start it up when you need to test more.
The most important thing about Amazon is that you need to set it up right. If you do it right, a big problem like Amazon had last month (4 days down) won't be a problem for you. If you mess up, it can mean disaster. So stick to what you know, and what works for you.
I recently moved one of my clients' sites to a RCS (in this case CentOS 5.5), and it's working out quite well. I have full control of the OS and packages, and do not worry about the hardware. For a small licensing fee per month, you can Windows OS options (although I don't know a lot of details).
It's scalable (load balancing, scalable RAM, etc), and it's inexpensive. Definitely worth a look here.
For a few more bucks, you can have managed Cloud Servers, etc.
I would suggest skipping the cloud, since you've elected to use windows. The better solution in my opinion is to keep your windows systems and get a CDN to cache and content accelerator your site (obvious plug, my company http://www.winkstreaming.com/en/wink_shield/) but there are many companies offering this solution at decent prices just google CDN.
You are doomed, at least in this endeavor. Welcome to the education that is leaping before you look.
All I hear is LAMP this and LAMP that because of cost. If you guys knew anything about what you are talking about you would know that Microsoft has a 3 year startup program for SaaS services which gives you everything you need to run a SaaS company including Azure access for a grand total of $100 that you pay at the end of the 3 years.
Going to the cloud is a decision which isn't reliant on the OS you are doing development on. I can write my whole frontend in knockout + jquery + jquery templates then simply build the service architecture in PHP or .NET or Ruby or Java with minimal effort. Real web developers these days are server side abstract. If you design a website properly, moving the server side architecture is relatively low cost and simple depending upon your business use cases.
If you want to stay with .NET, check out BizSpark and don't listen to these monkeys. I doubt most of them know how to write code/manage server infrastructures anyways.
"Java" could refer to Coffee indicating a complete lack of knowledge about software which aligns well with the post
Are you shitting me!? Java Man is FAMOUS! Everyone knows who he is, even has a Wikipedia page!
http://aws.amazon.com/free/
If you really believe in your idea, spend some time writing a business plan instead and attract some investors so you hire people to handle the details for you.
Learn to do it yourself.
Get it running reliably.
Troubleshooting is easier on equipment you control.
There is no substitute for knowledge and experience.
Once it is running smoothly, if the expected traffic actually happens, decide where to go from there.
Please , try not to do everything , assume your stronger position, and outsource the remaining , its also good to share responsability , so you don't go mad after a few months production is live.
How could you in a cost effective way ever hope to handle "millions of accounts and daily hits" from your home internet connection without paying through the nose? I mean I've heard of fiber connections that are pretty quick but, realistically you need redundancy to run a successful project. What happens when your one vm or your single internet connection goes down? At the very least you are best off shipping your server to a data center for colo and don't forget to backup often :-)
Because it's not a bad question. You're getting answers from a lot of people who are either so buried in the deeply technical side of things or locked into the past that they don't really understand it. Having shepherded a couple of dreamy startups through this phase, though, I don't think you are either crazy or necessarily under-qualified to make a successful site. To be honest, it no longer takes the hardcore technical skills that it once did, if in fact it ever did. Technical competency is over-rated by technical people; there are super-successful web businesses out there that started out (and sometimes even continued) with really shoddy coding and infrastructure setup. Craigslist and PlentyOfFish come to mind as examples. I know several others without such name recognition but which nonetheless did quite well for their owners, who slipped along with very basic programming skills and almost no hardware competency whatsoever.
Having the ability to outsource your infrastructure makes it even easier to do this today. I'm going to stay away from the "C" word because it's so shot through with marketing dross and misunderstanding now. But it's entirely possible to effectively do away with almost all the Windows admin if that's not your strength by going with hosted services. You are probably a long, long way from having to worry about Windows/LAMP stack comparisons even at your stated traffic goals, and using a hosted service will abstract that to the point where you aren't going to need to worry about it anyway. I wish you the best, but the reality is also that few sites make it big anyway, so while scalability is certainly something to consider at this stage, you shouldn't allow it to hobble your choices excessively. If you actually get there, it's almost certain you are going to have both the resources and the need to rip everything up and re-do the entire site from scratch once or twice along the way anyhow. You can re-tool then if you must.
Abstracting hardware doesn't absolve you from making other design choices that will afford scalability, and you should have some understanding of what's going on under the hood so you can make those appropriately. But you don't need all that to get started. I don't know anyone, at any skill level, who actually correctly made all the right choices on their first pass. You'll be learning along the way. That's actually an advantage; tech is filled with people who found their comfort levels and can't adjust to newer models.
It sounds like you are asking as much about your dev environment as production. I would say "yes," move it all to a hosted environment. At this stage, you don't need to be worrying about the underlying nuts and bolts. Get up and running quickly and easily. Be flexible and make adjustments along the way. You probably don't even need to go with a full-on PaaS provider right now, either; get a cheap hosting plan with a company that will help you scale when you need to. Depending on your service requirements, you can go with best of breed hosting to find the most efficient solutions for your various problems... use a SaaS vendor for version control, use a CDN for content hosting, and so on. It's cheap, it's fast, and it reduces the time and cost of failure. Failure is undoubtedly something you will run into a few times along the way. That's going to happen whether you are a technical genius or just some schmoe with a good idea. Build it in to your plans; don't over-invest (whether in time or money) in things until you can see better how things are working out.
If you have a good idea, don't be too afraid if you don't know what you are doing. A lot of the best people don't. One of the most valuable lessons you can learn is what not to spend time on, and a lot of things that certain folks here on Slashdot hold dear are things that you don't necessarily need to spend a lot of time on right now. Prove your concepts first. If you turn into the next Facebook, you can worry about infrastructure then. Until then, don't let the idea that you need a Facebook-worthy infrastructure before getting started to hold you back. Re-format that VM server into a games machine and go rent time elsewhere.
No relation to Happy Monkey
"If you can't be responsible for every part of your business, you don't need to be in business, PERIOD."
So a person making widgets should know how to run a server? Bullshit. He said he's creating a web site for his business, not that he's becoming a web site designer/host. He never said what his business was (presumably he thinks he's invented the next mousetrap.) That would have been helpful to give some sort of idea at least if this was a service or a widget you sell out of an online store.
To the question: I don't think you need "the cloud" but I think you do need to find a reliable web site host where you can start small, but have room to grow. Yeah I know you think you have the next hot idea, but it still takes a while for things to catch on. Get someone with experience hosting e-commerce web sites and shops, especially if you're selling a widget. If you're selling a service you'll need something more basic but which can handle a large number of views and basics like email forms for inquires. Point is, if you're expecting "millions of hits" this is not a do-it-yourself project.
Oh and btw make sure you not only pick a good .com .biz, etc. name, but take advantage of the email, i.e., Joe.Blow@mybusiness.com or sales@mybusiness.com. IOW, do not use Yahoo, Hotmail, Gmail, etc. I work with small suppliers every day and I still see a lot of those crop up--especially one person shops. Those emails are fine for someone looking for a job, but look damn unprofessional on a business.
If you're not a web designer (and you know whether or not you are), either keep it really basic, or get someone (not just your unemployed brother) professional (someone with a portfolio you like) to design it. Doesn't have to be fancy and make sure you budget. Once you decide on a design, don't keep going back and expect them to make changes for free. Start making changes and you will blow your budget.
Good luck.
If you've never been modded as "flamebait" or "troll," you've never tried to argue a minority viewpoint here!
The bandwidth needs alone will push you to the cloud (or to a coloc).
You say you're expecting milions of hits/day - if each user pulls down 50KB of content, and you get a million of those hits over a 10 hour period:
50KB/hit * 1 M hits/day / 10 hours / 3600 seconds/hour = 1388 KB/second
It'll take at least a 15Mbit/second upstream connection to handle that bandwidth which is hard to get on a residential connection in most areas of the USA. Plus the mean-time-to-repair on a residential connection can reach days, so if you have an outage, your site will be offline for a long time. And you probably don't have a generator (and even if you do, there's no guarantee that the equipment that serves your internet connection is also on generator), backup cooling (if needed in your climate), or someone to reboot your server or swap a failed drive when you're on vacation.
You can certainly get that much bandwidth delivered to your house, but you'll likely end up paying more than the cloud hosting costs and still won't have the uptime and reliability you'd get from the cloud provider.
Look at appharbor.com. Their slogan is Azure done right: you get ASP.NET 4.0 hosting, a Git repository, continuous integration and unit testing upon deployment, SQL Server instances, etc.. Basically everything you're asking for. Their backend is 'the cloud': Amazon EC2.
Their rates start at 'free', so there is no cost while you're busy getting your millions of accounts and hits. When that happens move to one of their paid tiers and go nuts.
so you want office workers to landscaping and Janitor work?
With a few exceptions (Google App Engine as suggested elsewhere being one) "The Cloud" is nothing but one or more VMs that you can start/stop/scale at will and pay for only what you use. Have you even looked at any of the "cloud" implementations?
If you can't manage one VM, how are you going to manage a dozen?
I do think Windows is a poor choice, but if you are locked into that, you can certainly get Windows VMs on cloud providers.
As for "VM vs Cloud", yes, go directly to the cloud (which of course is just a VM.) That is, if you KNOW you need to scale, don't go to a VM provider and then switch later to a cloud provider. Just start with the cloud provider so that you only have to learn the tools once. So, for example, pick Amazon over Linode.
Yes put it in the cloud. If you actually have some clever secret sauce then take a loan and pay somebody to do the stuff you can't because you should be in a hurry, otherwise just build it to scale to tens of users.
No.
You're a one-man company starting out. You've got a big idea, but you need to realize the basics of that idea before it will have a chance to become something more than a gleam in your eye. Thus, start out with tools that you have on hand and have complete control over. Realize the foundation of your idea, then scale it. Start small, and plan your migration to larger platforms well ahead of time. If it takes off, you'll be well-prepared to meet the demand. If it doesn't, then you've not over-invested your time/money.
As for "the Cloud", remember the old adage "don't put all your eggs in one basket". Everyone has "bad internet days"; don't put your ability to work on your project at even greater risk.
-SS "Teach the ignorant, care for the dumb, and punish the stupid."
> presumably he thinks he's invented the next mousetrap.
Watch out, he sounds to me like he might have invented the next Facebook ;-)
Everything I write is lies, read between the lines.
But wasn't The Rapture yesterday?
You say "To progress the site I need to set up version control, continuous integration, and staging" . . . in the near future, I think you will be able to buy all that with a few clicks in the cloud environment. A good example that is available right now is the Pantheon evironment, although it is targeted at only Drupal: https://getpantheon.com/platform
I know the founders of Pantheon and have worked with one of them. The demos they have been giving, and the experiments I did with it using the Beta trail codes they gave out, were very impressive. I would start any large multi-developer, scaling project on Pantheon right now.
Eventually there will be similar offerings for all platforms. You probably should not wait for that, however. I would avoid basing the project on Microsoft products if I were you, and I would set up servers on the Rackspace Cloud, keep the code in a private github, and set up a single real hardware server at the house or office that has enough a development environment on it be to fall back and have that machine also keep backups of everything.
Let me get this straight. You're expecting to have millions of recurring members but you've just started thinking about version control.
Also, by your current description it sounds like your project is still in the planning phase. I wouldn't worry for now on how to go from 0 to 1000000 users- unless you're the next Zuckerberg, those are probably just illusions of grandeur. How about starting by figuring out how to go from 0 to 1000 by next year? If your service is of enough added value (a tenner per month per user) you should be able to live off your project in about a year. Once you reach that point, and you still feel the need to go bigger, ask again.
By the way, I'd be very careful choosing MS as your platform of choice if you want go big. All of the big boys that serve millions of users do some flavour of Unix.
A cloud infrastructure will only work even remotely affordably if you ARE the cloud, & you can't do that on windows (at least not easily). You'll need a shitload of rackmounts, switches, a highly expensive dedicated line, & you can forget about getting in bed with Windows; you'll need Ubuntu Server as a MINIMUM, say goodbye to your pretty little GUI & your .NET. Seriously, a Win2008 VM? It's like you're a kid banging 2 sticks together to get teamspeak to work.
When the Amazon bomb happened. There is your answer.
Start ups start successfully with Linux, and move to Microsoft when they become fat and lazy, and the IPO happens, the movers take their profits and vacations, while the flunkies stuck at the office hire crappy MS IT people. Duh!
"I foresee each step of the way (setting up a domain; SQL-Server, etc) as a slow, risky process, and a big disruption to development."
you have been duped by microsoft.
Security is a slow and risky process that you HAVE TO INVEST IN,
It is not a disruption or a interruption. keep your patches up to date.
unless your code is portable and your going to mysql.
Stuck is stuck!
"Should I forget my VM server (it will make a nice games machine!) "
No you should use it more widely.
You should have it side by side with a VC machine, as well as a few Linux VMs for conversion.
You need to set up honeypots and put them outside the DMZ to show you where you are vulnerable.
ITS GOING TO HAPPEN! just be prepared.
If you had an amazon event, how long would it take for you to recover?
how long?
This is one of the scenarios where developing your product in Java makes sense. You can deploy your product from your dev box at home (running Windows) to a deployment in the cloud (mostly likely Linux) without having to re-code anything or worry about library gotchas (as a C#.NET to Mono switch would entail). Plus, all your tools and software infrastructure (Apache Tomcat etc) can be free, which means your growth can scale without software licenses constraining you.
Why are the editors pushing the crap onto the front page? If he lacks the ability to even answer their question about infrastructure I doubt their projections regarding adoption of this 'site' are anywhere near "millions" thus undermining the very condition which caused the question about infrastructure in the first place. Do you editors deliberately seek out this sort of crap or what?
Or, he's literally a Java Man and setting up SQL Server *is* hard for him and all the tech stuff is confusing. It's not GEICO you know...
It must have been something you assimilated. . . .
Assuming he is honest about his skills, it is possible that "Java" refers to the coffee or the island, rather than the language.
I call shenanigans. The username of this "Windows developer" is "javaman". He names the major Windows dev and SQL Server brands but doesn't mention Azure. He refers to "continuous integration" but pretends that setting up SQL Server is hard. Methinks this is a plant...though I'm not sure for what.
How To Register for KBC 5
PlentyOfFish.com became the world's largest dating website earning ten's of million$ per year. It runs on all Microsoft software. Until a few years ago it had only 1 employee and ran entirely on 1 web server and 1 database server. On his blog plentyoffish.wordpress.com the owner described how a single $2K Dell server running Asp.net could handle 1B pages/month. Here is an interview where he describes his business and the Microsoft tech that it runs on: http://goo.gl/9inUl
The username of this "Windows developer" is "javaman".
Methinks this is a plant...though I'm not sure for what.
Juan Valdez?
Forget all these market-speak buzzwords (cloud, my ass!) and rent a low end/cost Colo with an uptime guarantee and discount scalability option, your choice of OS and base config is largely a matter of personal preference.
But seriously you might want to go to your local community college and take some courses before you break yourself.
I killed da wabbit -Elmer Fudd
So you are asking if you should buy your own server or rent space on someone elses?
So lots of 'helpful' people can tell him to use Azure obviously.
I think the only people that read slashdot these days are Microsoft astroturfers..
Yes, and I want them to proofread my writing, too; outsourcing to a country whose native language isn't English to proofread what you wrote in English is like pizza-ing when you should be french fry-ing -- you're gonna have a bad time.
"Good, Fast, Cheap: Pick any two" -- RFC 1925
Ooops. Link doesn't have video.
Here's audio:
http://channel9.msdn.com/shows/IIS+Show/IIS-Show-10-PlentyofFishcom-and-IIS-6--Plenty-of-performance/
In 2007 "the site runs on Windows Server 2003, 31 million page pages/day - 2 million page views per hour at peak, 40-50K concurrent users – on ONE IIS 6 server at 65% capacity."
You should start by figuring out the cost of staying on your own hardware vs. a cloud service, as a function of the number of users.
If you use your own hardware, how much will you have to spend on the following, as a function of users: ...
* Bandwidth: As you get more users, you will need more bandwidth to your server.
* Machine costs: As you get more users, you will need more machines, disk, RAM,
* Cost of data loss: Do users care if their data goes away when a disk fails? If so, consider the cost of reliable disks, RAID, etc.
* Sysadmin-time: How much time will you spend maintaining the servers?
For a cloud service, the price of these things will be obvious. For a home-made setup, you will need to load test your machine to see how much traffic each server can support.
The nice thing about cloud providers is that if you suddenly get an extra million users in a day, you can send the provider some money and keep going. I promise that the first system you host will go down under load a few times, and users who see the service in a broken state may not come back when you fixed it.
Most comments on the thread will be jokes about using windows as a server. As smart-assed as they are, you should listen to them. In nine years of working on systems like this, I have seen only two that chose to use windows as a server platform. Both fell apart under load, because there were not enough machines. The high cost of licences meant the founders did not want to have more servers than necessarily, and they were not ready when usage went up 10x in two hours. By the time they fixed it, users stopped coming.
I too am building a website with millions to billions of visitors. I have spent quite a bit of time starting with the back end. I have a single script that calls 5 more. Each of those may call two or three more. I set up and build apache from source (with hardening modules), mysql from source (with hardening modules and patches), php (with pecl add ons, hardening patches ---suhosin--, hardening modules, etc.). Then Drupal (with drupal modules). I actively prevent side jacking, click jacking, clear jacking, cross site scripting, cross site request forgery, buffer overflows, sql injection, cookie poisoning, cookie forgery, and add extras like odbc drivers and clients (all from scripts). The scripts also configure the http.conf files, php.conf files, my.cfg files for php. I also do some firewall setups (IP tables). Its all running on Linux. The front end involves (as stated) Drupal, and I have a heavy investment in Zen/CSS. The front end is easy compared to the back end, and I don't mean to trivialize the front end, its where all the magic that people see lives. I have test routines to do performance tests, and use programs like nmap to look for holes (among others). And have other scripts that do database backups/recovery plus archives of the site. The idea is that I start at a co-lo facility, then as things pick up, I can drag scripts to any new machine, and have new servers up and running. A load balancer front end and I will have a multi-server site that is relatively secure. Security is an ongoing PITA, but I HAVE TO. I've also looked at people doing packet injection by forging packets by examining border gateway protocol (and doing man in the middle attacks), but I'm pretty certain Linux has anonymised the packet sequence numbers so that injecting packets will be difficult. My site also sets up for multiple sites from one IP address using virtual hosting. Again, more work, but IP addresses are becoming scarce (well IPV4 anyway), and running several sites from one location makes life a bit easier.
Yes, as a one-man startup you should certainly run your production environment in "the cloud" rather than building your own infrastructure, because it's faster, cheaper, more flexible and easier than building it yourself.
That being said, the way your question is written, you are likely to need a lot of help doing so. Asking on Slashdot is a good start, but to go further you should find someone who's built a system that has scaled to millions of users, and have them help you. You're not going to get kind of extensive help for free from Slashdot. To say the least, there's a lot more to scaling an application to millions of users than choosing between cloud and dedicated hosting, and it's important to "get it right." One important thing when planning your time is that "building your app" is less work than "running your app in production" and "scaling your app to millions of users". That is, if you think that (not having built a scalable web app before) it'll take you three months to build your app, it take at least three more months to get it to run in production properly (with monitoring, alerting, failover, etc.) and at least that long again to get it to scale well. And since you've not done this before, it'll take longer than that (or find an expert and listen to him).
Where are you located? You should find someone local that's run something at scale, and do some white-boarding.
Could all the people calling this person a troll, and "domyjobforme"-idiots just stop wasting others and their own time with these ridiculous negative non-helping no-good-for-anyone answers ? Who are you people anyways ? Why bother with answering ? What do you get from posting such answers?
You are so awfully clever on others behalf that it sickens me.
Give some proper answers - his a fellow nerd that asks perfectly legit questions to a community of nerds. It doesn't matter if he is out of his league, a new bee or a professional for that matter. He asks for advice, and those kind of negative answers is a waste of everybody's time and energy. Pardon my French, but go post your stupid answers somewhere else...
I still remember when the service needed to move a website to the Internet was called web hosting.
"The Cloud"... Give me a break.
"LOL, so you've never called in an outside contractor for ANYTHING?"
I don't know how you managed to extrapolate that nonsensical statement from what I said, is English your primary language?
My parent company is a full in-house production company. Yes, we do happen to do everything. If we have a need for a skill, we simply hire someone with that skill and get them to work.
It saves a ton of money and ensures that we get competent people.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
"Get someone with experience hosting e-commerce web sites and shops, especially if you're selling a widget. If you're selling a service you'll need something more basic but which can handle a large number of views and basics like email forms for inquires. Point is, if you're expecting "millions of hits" this is not a do-it-yourself project."
Slashdot hasn't managed to bring my website down yet, nor 4chan and their pitiful LOIC.
Certainly didn't take me much to figure out how to do that - don't rely upon other people.
The old adage goes - if you want something done right, do it yourself.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
Why not? I run every aspect of my business from property maintenance to product specification and design.
Why can't they?
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
I have to think this is a joke.
Running a serious business server using a Microsoft product means you will HAVE to have a full time staff and multiple redundant systems to even approach a decent availability of service number.
Sounds to me like you would do better with GoDaddy or some other simple, easy webhost to avoid straining yourself.
Linux computers, watercooled, photography
Go straight to Azure. It is easy, cheap (really cheap) and Microsoft will give you lots of help to get strarted.
We have done it and it is easy!
Disclaimer - We are in the Microsoft ISV Partner program.
If only everyone was as great and capable as you. I'd much rather delegate. And I do run things in the cloud and it's great. It is far cheaper than building up infrastructure. I don't put everything in one place though. I've got my services up on multiple cloud platforms. Personally I think you made a bit to bold of a statement and are too proud to admit it. Join the real world here. First of all you don't do everything yourself. Maybe your company thinks it does but hiring employees isn't that far removed from hiring another company. In house servers are just as risky as the cloud.
Or maybe he really likes coffee
Certainly didn't take me much to figure out how to do that - don't rely upon other people.
The old adage goes - if you want something done right, do it yourself.
At the risk of using business cliches, I think it's better to focus on your core business than to try and do everything yourself. Otherwise you usually end up doing a lot of things badly.
You say don't rely on other people--that's not only sad, it's impractical. Because eventually, hopefully, he'll have to hire some employees...
If you've never been modded as "flamebait" or "troll," you've never tried to argue a minority viewpoint here!
This smells like a troll, therefore it is a troll.
I suggest you take a look at Windows Azure. http://www.microsoft.com/windowsazure/
Listen, we're a LAMP shop, RHEL and/or CENTOS all the way! We've been in business for a decade, and currently have a 20-node cluster for our $2 million/year niche software company, averaging about 20%/year growth, even with the down economy.
And although we're a LAMP shop, I can tell you that hardware/licenses are a tiny part of the costs of our business if the application is structured correctly. ONE of our clients had a dozen servers before switching to our system. Our cluster of (at the time) 6 nodes didn't even register the increase in traffic meaningfully.
It wouldn't have been much different had we gone with Windows Server vs Linux, both can be viable. Licensing costs are a small fraction of the total! There's administration overhead. Power. Bandwidth. Backups. and on, and on. Administration costs dwarf your licensing costs!
Don't avoid Windows to avoid licensing costs, go with Linux because of the fairly significant reduction in Administration costs! Even with 20 nodes and hundreds of clients, thousands of users, we still have only a very part-time admin. (me)
I have no problem with your religion until you decide it's reason to deprive others of the truth.
I challenge you to find a single web startup that has paid employees and also has licensing costs of windows taking up any significant percentage on the expense sheet. Just One. Cmon.. ;-)
>Methinks this is a plant...though I'm not sure for what.
man are you behind the times! This is not the shill of 5-6 years ago. Marketing has evolved.
These days, it's not a plant for SOMETHING. It's not designed with an end-goal in mind. You're not directing traffic somewhere, you're not getting somebody's name out there.
No. You simply go, and you babble. Think of the difference between plants today and shills 5-6 years ago as the difference between the guy hustling for money, and the raving lunatic who isn't asking for anything from anyone, just running around throwing feces all over the place.
Do I understand the Internet anymore? No.
and that makes you what?
I find this business model of yours intriguing. A fully self-sufficient business sounds like a great idea, but I am not sure my business can afford the capital outlay. How did you manage to buy the oil reserves to power your company's vehicle? Or am I assuming too much? Since your company designed and built all your motor vehicle in house, they could run on anything, right?
If it's in you sig, it's in your post.
I don't know what your business is and would have no idea what the best platform is if I did. What I do know is that getting a product or service out there is more important than it being perfect.
Lag shouldn't be even a part of the consideration of "where" to host it. If he's looking at development and testing efforts, they'll all be on local servers (probably VMs on his machine.) Bandwidth, well, if he's got millions of users he's not going to be running them all over a DSL modem.
I think the real answer is: how much effort can he spare from his main task of developing the application to maintaining his own hosting site? He's a one-man shop. Can he afford to divert X hours per week building a server farm, hiring electricians, installing HVAC systems, and once it's running doing the maintenance tasks like recovering from disk errors, doing backups, etc? How much would that slow him down from getting his app operational? Remember, his app won't make him a dime until it's fully up and running.
A hosting provider takes a lot of that crap away - for a monthly fee. To get his first system up on a shoestring budget, he could self-host on a PC in his basement. When it's time to grow, he could consider expansion to a hosting provider, and if his site is as wildly successful as he's imagining, he could build his own data center next year once the cash is pouring in the door, and he understands how big his systems will need to be.
Self-hosting will suck if he's as quickly successful as he thinks: he won't be able to simply drop in a dozen more boxes into his basement. With the cloud, you simply write your provider a bigger check and the resources are there.
John
Here is the honest truth. The cloud is nothing more than an abstraction layer between you and the physical hardware that is shared by hundreds if not thousands of other users.
By placing your entire infrastructure in the virtual shared hosted networks you immediately lose all control over how your services function.
Loss of control leads to loss of organization, loss of reliability, loss of redundancy, loss of performance, and loss of security, among others.
Reality:
1) Virtual shared hosted services are perfect for front ends that have the built in capability to fail. They will fail no matter what. You must build that into the design.
2) Backend databases should NEVER be placed outside your direct sphere of control, control MUST include layers 1-7.
3) Do not throw all of your services into one provider. If you're going to use virtual shared hosted services then get a few instances in all of them.
4) ALWAYS assume that a SaaS/PaaS provider is hosted in Amazon EC2. From email filtering to web filtering to all these other social networking service providers. They are all in AWS. See #5
5) Amazon EC2 is a complete disaster and should only be used as a DEV/Stage environment. The other services AWS provides are better but should only be relied upon by in a tertiary capacity.
6) No really, Amazon is NOT a good place for production data. See #5
7) Virtualization is key, control of that virtualization is even more important. (See Openstack and VMware vSphere)
8) See #5
Always be able to see major shifts in computing with the underlying technology and not get sucked into the marketing storm that comes after as everyone jumps on board.
Assume everyone is trying to get money out of you.
If it sounds too good to be true IT IS, especially in "the cloud".
I've used Virtual Private Servers (VPS) before, and agree that they are the way to get started. You get full access to the server, and so long as you buy a mid range package you should have ample power for a startup project. If your website traffic explodes then you can look at moving to a large dedicated server or to a cloud provider, but unless for the majority of websites those would be overkill.
I've written a blog post about VPS/VM vs Dedicated vs Cloud hosting providers before, coming mainly from an ASP.NET point of view. The bottom line is that you have to find something that fits your budget. And don't even think about hosting it yourself. If you really need to be able to scale quickly then Rackspace Cloud, Windows Azure, or AppHarbor are all viable options, but with the exception of AppHarbor they also all have a big up front price tag.
http://appharbor.com/ - AppHarbor has a git based continuous integration workflow that will automatically check out the code from your repository, build and run unit tests, and if all passes, deploys your application to their service. Their website is rather sparse, so you might want to Google for more info.
If we have a need for a skill, we simply hire someone with that skill and get them to work.
What happened to
If you can't be responsible for every part of your business, you don't need to be in business, PERIOD.
Cloud services like Rackspacecloud and Amazon EC2 are probably going to be more reliable than whatever you're planning to run yourself. They're straight-forward to learn, scale quickly, with relative ease, at reasonable cost. I don't see "going to the cloud" as an extra hurdle, but as a way to reduce headaches and to simplify deployment from the start. If you're worried about cost.. you can start with Linux instances at Rackspace for like 11 bucks/mth. On the topic of OS, either Linux or windows servers will work just fine, and if you're not comfortable with Linux, I wouldn't mess around with it unless you really need to save every penny. Linux instances give better performance:cost ratios than their windows counter-parts, but (in my experience, windows guy here) take more time to set up. If your project starts to get lots of traction, and you can start to pay yourself for your time spent on it, then it would start to make sense deploying real servers, or to start learning how to administer Linux instances/servers. Anyhow, good luck with the project, have fun! :)
You're still responsible in the sense that if you don't hire in the correct person, properly communicate the requirements to them, etc, then it's your ass on the line.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
go to the cloud. I'm a unix greybeard and have set up entire ISPs using Linux and Unix and right now my blog is sitting on a Bluehost box running Linux. Why would I co-locate a server and pay bandwidth charges and then worry about back-ups and hardware failures when I can do it for $6 a month at a hosting service?
For you the question is even more pertinent since you are clearly not competent to even decide on an operating system.
Once your web site is actually getting the kind of attention think it will then you can hire engineers and administrators and move the site to your own machine. But here's a hint: Don't put it on Windows!
No one ever had to evacuate a city because the solar panels broke!
plz MOD PARENT DOWN
parent post is not only an obvious troll, but the attitude is poisonous for our profession...is he trying to say he never asked any questions when starting to implement a new skill? If you know everything about how to do a job, then by definition you're not a beginner. BEGINNERS MUST BE HELPED NOT SHUNNED!!!
i'm a CCNA, php developer (mostly in Drupal these days) that picked up web coding later in life (after years of database admin and music writing)
Thank you Dave Raggett
That's also what contracts and SLA's are for. Just FYI, Azure/Microsoft offers SLA. So does any other good company offering services for businesses.
Come on mate. Apparently he looks good in spandex with his cape fluttering in the breeze and his underwear on the outside.
Science advances one funeral at a time- Max Planck
"Cloud" is vague.
Microsoft, new, (3 yr old) privately-held company with less than $1M in revenue? Go BizSpark, and heavily consider leveraging Azure.
In general, cloud/SaaS equates to rental, and right now, in the long run, it's still cheaper to own your datacenter, if your needs are extensive and not highly elastic. However, cloud/SaaS also equates to low cost of entry and incremental costs to expand, as well as outsourcing of non-core proficiencies you probably don't need to waste time or money on right now.
I'd recommend any startup go cloud/SaaS where it fits their budget and needs and growth expectations, while keeping an eye on longterm goals/costs/capacity needs, because at some point, it may make sense to bring things in house. Then again, netflix hosts their content with CDNs and their website with Amazon AWS, so to some, wholly outsourcing the datacenter is apparently a feasible solution.
--ab
I am, therefore you think.
(My background: I've been doing IT for the last 18 years or so; for the last two years, I've been working at Netflix, one of the highest-visibility cloud consumers out there. Until two weeks ago, I was on the IT side, focusing on the datacenter; about two weeks ago I moved over to Cloud Operations, focusing on the cloud (duh) and monitoring, specifically. The following is my opinion only, and does not necessarily represent the opinions of my employer).
In my opinion, the cloud is the easiest way to launch a new service with reasonable redundancy and growth potential. It's how I would start off any new business. Have there been failures? Sure. But largely, cloud failures have only impacted cloud consumers who engineered their environments in a non-fault-tolerant way, in the mistaken belief that "the cloud never fails." The cloud fails. It fails all the time. But following good design principles (ideally, be in multiple regions; at minimum, be in multiple availability zones; test what happens when an AZ dies an ugly death) will give you better uptime, with better cost, than you would achieve for a reasonable amount of money running your own datacenter systems.
And then, once you've got a significant enough size with a big enough ongoing consumption of cloud resources, you can look at creating your own DC environment.
Which is why, if you ever suspect you have a tumor, you should get out your scalpel and do a biopsy on yourself, look at the cells under a microscope, and decide what the next step should be. And if you ever get accused of practicing medicine without a license, you should defend yourself in court.
Thanks for all the comments.
I put on a very thick skin before posting, so the "you can't be serious" and "you are obviously not qualified" comments have not bothered me.
Having been on Slashdot for some 10 years, I expected that (if my question was posted) that I would get some useful ideas from left field, and also some useful direct comments, and that is what has happened, and I am especially grateful to all who took my question seriously and shared of their highly valuable expertise.
One bit of info was unfortunately omitted from my question, because of finger trouble while I was editing it. Obviously I am using Microsoft products, but it's not just because it's all I know (it isn't) but because I have used .Net for 8 years, and find myself always enjoying the latest stuff (now Linq, ASP.NET MVC, and F#) and, as a developer, this enjoyment is central to my productivity. I would never tell a RoR developer to stop enjoying his stuff, and I give myself the same license.
A brief comment on the useful Win vs Linux comments. There has been some advice that, were I to switch to Linux, that I would find a Linux network easier to manage, as well as obviously saving money. I am also "barely competent" as a Linux admin, but I can see the wisdom in that, and I have found Linux administration much more enjoyable than Windows. If I had time to spare, then I would probably go with Mono, for the "best of both worlds", but that's not realistic.
A lot of the advice relates to the cost of scaling if the application takes off, with warnings about the costs then of Windows. That is a factor I had not considered. I am, because of my free Windows infrastructure, immune to the costs for development, but I will have to look at the costs for scaling. I had thought that, if I'm getting many hits, then I would also being generating sufficient revenue to pay for servers, or that I would have sufficiently proven my concept to get investor funding, and possibly switch to Mono then. IOW, that would be a problem I would like to have. The comments here have suggested that I reconsider this optimism
I haven't yet processed all the most helpful comments, and I will certainly be taking my time to do so, but there does seem to be some repeated advice that Azure is the way to go. This will, hopefully, provide a uniform environment for development, staging and scalable hosting.
Some advice to young people out there. Don't be afraid to ask stupid questions. The chances are that your question and the answer will save you a lot of time.
Thanks again, Slashdot
And, a quick comment on my username. I adopted "Javaman59" when I moved to modern OO development in 2001. I have kept it, with some fondness for those roots in Java, Linux, and the GOF patterns.
Javaman59
I'm a software visionary. I don't code.
Or his username could be referring to this "Java Man": http://en.wikipedia.org/wiki/Java_Man ;)
I'm not sure I understand the question. It seems the guy is just asking whether he should rent space from Rackspace etc. , instead of physically building and self-managing a web server. Wasn't this question around long before "the cloud" buzzword came in?
Thankyou Sir! I am pleasantly surprised that someone took my seriously so quickly.
I was not aware the VS 2010 integrates so well with Azure for development, so that is probably the most useful bit of info I needed
Unfortunately, my question did not make it clear that I am asking only about developmental and staging infrastructure. I never expected to use anything other than a commercial hosting solution ("cloud" or otherwise).
I'm a software visionary. I don't code.
Why not, let's go with a car analogy.
Most people actually have a fair idea about car basics. They know about mpg. They know about airbags, and four wheel drive. They know the difference between gasoline and diesel. And importantly, they know how to drive.
On the face of it, this sounds like a person who doesn't know any of the above, asking whether he should buy a car or rent one.
And to top it off, he wants to start a trucking company!
There's nothing wrong with renting servers per se, but if you want to run a website, you'll need to know how to run a website. He'll have to either hire someone with technical experience, or get some himself.
Frankly I'd just bite the bullet and get a Linux/OSS stack running. It's cheap, easy, well-documented, you can host it anywhere, and just about any basement nerd you can grab hold of will have experience with it. Who the fuck ever used Windows Azure?
Host your web site in VM on your home box.
Start a company all by yourself all the while having no experience in any function necessary to run it.
Write security-sensitive applications while being at the skills level when your choice of IDE is in any way relevant and can affect (positively or negatively) the result of your work.
I also recommend drinking a lot of bleach.
Contrary to the popular belief, there indeed is no God.
April Fools! Oh wait..
You want it to scale and you are using Microsoft? Aren't those to mutually exclusive things?
*gasp* slashdot got trolled? Say it ain't so!
would add something as version control and build infrastructure as an after thought. Give yourself and everyone else a favour and upgrade to something designed for the kinds of problems you are trying to solve like Linux.
As the island of our knowledge grows, so does the shore of our ignorance.
There are plenty of lightweight email servers on every platform. Some are very simple and have portable mail storage formats you can even read in a text editor if you have to. For nothing but email Exchange is very well named - it's better to swap it with something else. It only ever becomes remotely worth it for the calendar and other assorted bits bolted onto it, so if you are not using those it is very much the wrong tool.
What a bunch of haters. He (or she) posts a question. Admits what knowledge they have and don't have. And in return, everyone rips em a new one. "The question lacks any signs of expertise". "What is this on Slashdot for". 'why are you so stupid for going with going with ms'.
The poster has an idea and wants to run with it. And is asking /. colleagues for their opinions. Seriously, I think so many of you only get through the day by putting others down.
If Tesla posted a question about resonant transformers, many of you would discard him for not using your favorite dielectric in the capacitor. All the while, missing what he's trying to accomplish.
First, you need to learn. Personally I would go with Linux and open source, but do whatever you know. If you don't know, then learn. I have a dual quad core Linux Server w/ 8gb of RAM colo'ed at a data center and I can easily hand millions of hits a day using Lighttpd, Varnish cache, Memcached, PHP APC cache, PHP & My SQL. Second, it will take a long time for you to get millions of hits. It doesn't happen overnight, so you have plenty of time to learn about systems architecture & design. You could easily start with a decent sized 4gb RAM VM running Windows 2008 & IIS. Then, you can scale that up to a larger VM with 8 or 16gb of RAM. After that you need to design your system to scale out using multiple VMs and then you could also start using a Content Delivery Network like Akamai. You will learn as you go, you have plenty of time, trust me.
Or his username could be referring to this "Java Man": http://en.wikipedia.org/wiki/Java_Man ;)
Close :) It is a palendrome... I mean a pun.
I was a java developer some 10 years ago when I joined Slashdot. I have nothing but respect for the Java and LAMP world - obviously they have been proved over-and-over again.
I'm a software visionary. I don't code.
You're supposed to forget how they dropped the ball, and think all the cool people are using the cloud.
I wouldnt choose that os 15 years ago or now, can we even ssh to a microsoft box from a default install yet? Oh thats right rdp, thats really fun to automate. Let me guess theres a 3rd party app that can automate rdp and record some funny ms session. Dam this topic has got to be a joke lol!
Where does your company do its hosting? Who carries the data? Are you laying fiber to the rest of the internet? Are you sure you don't rely on some SLA to get your data out? How about delivering physical goods? Do you carry them yourself or rely on FedEx/UPS/etc? At some point its not economical or advisable to perform every last aspect of your business in house - a good business will evaluate what is core to getting its mission accomplished and contract out the rest. For a lot of startups, such as the one in the story, its not a good idea to put out a lot of capital expenditure just to test out or get a potentially non-viable service off the ground. As long as you're doing it intelligently hosting "in the cloud" can save a lot of time and money that would otherwise be tied up not delivering your product.
Just go for it. Don't let technological niceties get in your way. If you can get it to work and you are comfortable some non web friendly language such as ASM, cobol, fortran just getting it done still puts you ahead of most people with an idea. Worst case scenario you will find that as the hits pile up you have to redo the slowest bits. The funny truth is that for even medium sized sites the servers rarely cost even a tiny fraction of your revenue. Up to a fairly good sized site with a bad architecture you can just throw a monster 8 CPU zillion gig system at it that will keep it alive for a while.
As they say those are good problems. Too many great ideas spent their time being perfected and not released. But many good ideas were badly built but quickly released and later switched architectures. Twitter is famous for their Ruby to Scala switch.
For anyone who looked through the gawker code (love them or hate them they are big) it was crap on a stick. They waited too long to fix it and paid the price. But it would have been a greater price if they had made it perfect before releasing and then missed the boat.
Where you have to be careful with the cloud is that a burst of hits results in a burst of revenue. Otherwise you will just be left with a burst of costs. I can't imagine what a slashdot hit would cost on amazon.
Good luck.
So what? He has UID 524434 so he has been around on slashdot for a quite while. That just shows he has past coding experience in Java, and quick google query shows he is coding with C# now. Java->C# is a natural progress (as the languages are similar, but C# is better) and Visual Studio 2010 and Windows environment makes a lot of sense for C#.
You are 100% correct. I keep my Javaman handle because I've already got it, and also because I remember that phase of my career with respect and gratitude. Nevertheless I now use F#, C#, ASP.NET MVC, because I find them more enjoyable and productive. Of course I then use VS 2010. But I am a developer, not a sysadmin, and with this question I am looking for the minimal way to get my sysadmin done. Unfortunately, many have interpreted it as a question about the hosting, rather than developmental infrastructure.
I'm a software visionary. I don't code.
Go straight to bankruptcy.
I call shenanigans. The username of this "Windows developer" is "javaman". He names the major Windows dev and SQL Server brands but doesn't mention Azure. He refers to "continuous integration" but pretends that setting up SQL Server is hard. Methinks this is a plant...though I'm not sure for what.
shenanigans! I better go home and get my broom
Your "physicist wants to measure gravity waves", and not having a clue, is a very bad example. They know perfectly well how to try such a thing. In the past it was, a very large mass with a very finely tuned resonance. Problem with this was, you chose the frequency without being certain it is a common gravity wave frequency. The latest gravity wave detector is LIGO. A big cross shaped thing, you send lasers down a 4km arm or so, and then interfere them when they get back to the start. Gravity waves change the length of the arms, and this causes a change in the interference pattern. IIRC, the resolution is a bit better than the width of a proton.
3,200,000 Kb ram just for thread stack space.
512Meg my ass.
40,000 per given 5 minute period, maybe.
not 40,000 per one second interval.
Instead of having that pizza for dinner, buy 4gig of ram dude.
I find it ironic linux coders/hackers make their own scripts/setupts, yet FAIL at even making their own pizza, but buy a ready made solution for food.
Liberty freedom are no1, not dicks in suits.
The Cloud is not universally feasible or well advised for every business model/plan. However if you do your business plan (you doing a business plan, yes), an answer should become pretty self-evident pretty quickly.
in house has less lag and more bandwidth then working over the web from your home. How good is your upload?
My upload is slow. I've already discovered this using skydrive for some of my documents
You have addressed one of my concerns in the in-house vs.. cloud question, namely the development experience. I am wondering whether it is better to "bit the bullet" and master the in-house sysadmin, than put up with delays for check-ins, file comparisons, debugging, etc... I think your advise is that this is a valid concern
I'm a software visionary. I don't code.
That is the best advice someone can give you - makes an enormous amount of sense!
ScuzzMonkey, you have my respect - wise words. :)
then you're already half way to failure. A buzzword does not equal success and will not eliminate the need for hard work. Welcome to the real world.
- I've got bad karma because I won't parrot everyone else's opinion
I'm part of Bizspark (free) and that gets me 2 and a little bit more Azure instances running for a month. I find deploying to Azure more seamless that a web deploy (the VIP swap feature means zero downtime of your app).
There is a lively group of Ruby on Rails programmers in my neck of the woods who like the scalability of Rackspace.com for hosting. You might want to check them out- it is a grow on demand infrastructure (that is what you need) I recently installed a Windows SBS 2011 server - It was bundled on a Dell server with a quad core processor and 12 gigs of RAM. This hardware is spec'd per Microsoft for 75 users. So if you really think you are going to be getting the SQL traffic you outlined above, a Rackspace.com or cloud service would be my only recommendation for hosting. I supported web hosting for 10 years. You are lucky to be coding now, 10 years ago the server above cost $40k from SUN, now these things sell for under $2k and the cloud model is perfect for what you describe. Good Luck
As someone who develops such sites for a living, I would suggest to keep things simple and just get the minimum hardware/hosting you can get by with. Time/money you spend trying to get your website scalable before it is necessary could be better spent getting and keeping paying customers. Once you have the demand, then you'll have a justification for 'moving to the cloud'.
Any kind of automatic cloud scalability is going to add substantial complexity and maintenance requirements. Unless a key selling feature of your service is scalability, I would not build for it in the beginning.
Use TortoiseSVN with beanstalkapp.com for version control, continuous integration, staging, etc. It is fantastically easy to use. You can set up an automatic deployment to a staging server on each commit, and then do manual deployments to your production server.
Another tip, you'll save on hosting costs if you go with a PHP/Linux/MySQL stack instead of Windows/ASP.NET/SQL. A single dedicated server at host like pair.com can probably host millions of hits on a carefully tuned application, and they throw in free/discounted hardware upgrades (with almost no downtime and no administrative work on your part) every few years. I could not be happier with their service.
""I am a one person company developing a web site from home. The site is hoped to attract millions of accounts and daily hits"
User is living in Cloud Cuckoo Land already.
Go to Amazon and measure the cost-per-hour of a Windows server against that of a CentOS server. Microsoft in the cloud will hit you in the wallet pretty hard.
Some background on what to expect at as you scale:
http://www.codinghorror.com/blog/2009/06/scaling-up-vs-scaling-out-hidden-costs.html
I don't recommend Windows for a web startup, but if it is what you know, then it is what you know. I'd say the answer to your question is yes. As a one man show, you won't have time for much IT work. It is a poor allocation of your limited time.
Here is a simplest solution, but as always there are cheaper competitors:
http://aws.amazon.com/ec2/pricing/
#6495ED - cornflower blue