Simpletons guide to web server analysis
All of the sudden, by bandwidth has gone up? I wonder what people are doing on there?
All of the sudden, by bandwidth has gone up? I wonder what people are doing on there?
For my band, for my band...
Would any of you care to help out with putting together yet another online community? The aim with this one is to try and organize Railroad Earth's online presence just a bit, mainly so that newcomers can have a better central clearinghouse for RRE info and community before swimming into deeper waters. This probably isn't the most expeditious place to post this, but I'm trying to keep this kinda small while we're hammering out the design details.
That said, if you're an RRE fan, or a curious onlooker, please check it out here, and feel free to let me know of any annoying settings that you discover that I might be able to change.
As in, what does that mean, anyway? I'll try and keep it short.
When I spoke of Ruby on Rails the other day as the 'hotsh!t dev platform of the day', I realized that it would need some clarification. So let me clarify.
Coding pretty much anything is a massive, time consuming job. That's part of the reason that computer languages have evolved from 1s and 0s to Fortran to C to C++ to Java to Ruby: if you were to try and write the Twitter website in assembly language (binary), you'd be there coding for a very long time. If you were to try and write Windows 7, you couldn't do it. Hell, they can barely do it now.
Man has devised a better way. I wrote about some of it in the Database post. He has set to automating as much of the menial, repetitive work as possible so that the developer may set his mind toward accomplishing other tasks, like the fun bits of the application. In the case of iPhone development, Apple has created an environment in which to write the apps, X Code. X Code is itself an application that runs on your Mac just like any other application. The nice thing about X Code is that Apple designed it specifically for the purposes of making applications for Macs and iPhones.
History lesson, culled mostly from the Mac dev bible, Cocoa Programming for Mac OS X:
Apple computers was started years ago in a garage by two guys named Steve. They're both very famous now, Steve W perhaps slightly less so. The two guys named Steve decided that since they were both programmers and engineers, that perhaps they should hire a CEO type to help them with their rapidly growing company. CEO guy came in and moved Steve J to some kind of powerless position in the company, so he split. He started another company called NeXT. They never did anything commercially successful, but they did develop an operating system and a platform for writing applications – one that started with a fairly significant library of code already put together to make the developers job that much quicker. At some point, Apple was not doing well financially or otherwise and Microsoft came in and basically gave them a lifeline. This was in the 90s. Steve J at that point was an intensely innovative failure. Funny, huh?
Around this time Apple decided to buy NeXT computers along with the Operating System that they had developed. Steve J was brought back as an advisor for a little bit before becoming CEO within a year. The NeXT OS eventually became Mac OS X, which eventually became the iPhone OS. The point is that even though the iPhone has only been around for less than 2 years now, the developmental lead-up took decades. During that time, various developers decided that they would put together 'libraries' of code for making Macintosh applications and save themselves the time of having to rewrite all this code that's essentially “boilerplate”, meaning the same in every app, there to provide low level functionality that the end user never encounters and is very rarely altered even by the developer though widely varying apps are coming out the other end. Every language that's in any kind of wide use has it's own set of libraries, there to provide standard functionality for the programmer and standard sets of instructions for the computer to execute when certain functions are called within the app.
Okay, sorry. There's a lot in there that I wouldn't have understood 9 months ago, so let me break it down for you.
When you tell X Code to start a new project, it asks you what kind of project you'd like to start. It gives you the options of a couple of different layouts of iPhone apps. Perhaps you'd care to build a navigation-based application. That's when you use buttons to navigate through different tables of data and settings. The Setting app on your (or your friends') iPhone is an example of a navigation-based app. Maybe today you'd like a tab-bar based app. This is like the iPod app on the iPhone. You have a menu of a few pages at the bottom of the screen that take you to different pages in the app. Maybe you just want a blank window because you're building a game that your crew can play in the bus between gigs. Anyway, after you decide that, it puts together the skeleton of your application with a bunch of different chunks of code that every single app is going to need to run. The developer then fills out the skeleton and a newborn baby application is brought into the world. Apple calls their development framework Cocoa, and for iPhone it's Cocoa Touch. Cocoa is just a collection of pre-compiled libraries of code lashed together to help make the dev process that much easier. Cocoa has a name, so it seems like it should be a physical thing. It's not. This has been one of the hardest parts about this process to get my head around. It's a bunch of code that someone else already wrote so that you don't have to.
Ruby on Rails is also a framework. A big part of writing Rails apps is using a command called 'script'. You call a script command to help you build these chunks of code, and the script automatically does a LOT of the tedious work for you. I love this, because I don't have nearly the experience to write something from the ground up, and that sounds like a lot of wasted time besides. Tweaking the generated code is then how the developer builds his app. Brilliant. This is what a development platform is. This is why most Mac applications have that “Mac-y” look and feel. This is why all these New-Web social networking sites have a that sorta clean, modern look and feel, and why so many of them employ Google maps as well as each other in their functionality.
Once you get your head around which platform you want to develop on (I'm thinking Rails to learn), then you can start learning your way around the API, which is where the real fun begins. I'm not there yet, but will try to remember to link there when I am.
#general-development
This part of the blog is going to be tough to write. Not because I don't have much to say on the topic, but because I have frequent pangs of “dude, you don't really know anything about any of this stuff”. Then the other side of my brain speaks up. “Dude, it's cool. They know you're not a professor. That's the point of this whole thing, right? If someone wants hard facts and a professionally written tutorial on computer science, they'd probably go somewhere else. If they're here, they're either a friend or an understanding onlooker, and either way they're probably well aware that you're just a bass player. Nobody really expects that much from the bass player.” So, I'm on I 70 headed east across Kansas after selling the shit out of the Ogden theater this weekend.
And now, my initial impressions on the theory and practice of the database.
What the hell is a database? Being a visual kind of thinker, I've been trying to come up with some sort of relatable analogy to help myself understand exactly what it is.
“It's a series of tables, populated with data, and the tables relate to each other.”
Okay. What do you do with it?
“Well, once you have the data in your database, you can run queries on it.”
That sounds like a lot of fun! But what the hell does that mean?
“Well, queries are like a request that you make. Say you have a database that's populated with every song that RRE has ever played at every show in every venue. If you wanted to know what you played at a given venue the previous time that you were there, then you could query the database to give you the set list from that show.”
Well, that's cool.
“Yeah, but say you wanted to know how many times you've played Mighty River. You could query the same database and get an answer on that, too. Say you wanted to know the venues where you played it. You could query that, too.”
Okay, I get it. So a database is kind of like a big spreadsheet?
“Sort of, but not really. You see a spreadsheet is basically one big monolithic sheet that stores your data in rows and columns; like a database, but a database is made up of many of those sheets, and the collection of those sheets and the way that they relate to each other, and the queries that you can then make of the data is what makes it a more useful tool than just a big sheet with some numbers on it.”
Okay. So, how about an example?
For this, we shall turn to the web. Actually, never mind because this is a blog which means that you don't need to turn at all. Let's press Ctrl-T and get a fresh tab going in your browser and go on over to Amazon. Mac users have the option of Cmd-clicking the link to open in a new browser tab, so you don't have to navigate away from me.
Okay. You see all that stuff on the page? Just about everything there is stored in a big-ass database – all the text, all the ads, all the pictures. Now, why is this important? For that, we'll turn to my bookshelf and it's copy of The Complete and Unabridged History of The Internet. I see here in chapter two after all this non-sense about universities and the DoD that once upon a time there was no such thing as the World Wide Web. I'll skip to the part where then there was a WWW. The first websites on the newborn baby web were mostly authored in straight HTML, an interpreted language used to help your computer assemble some data on the other end into a web page. A skilled HTML coder can control every single element on the page that's being displayed down to the very pixel (until it's opened in another browser, but that's another chapter). The HTML basically tells your browser what text to display, how to format it, what the background color ought to be, where to place the pictures and to what they should link. This is understandably tedious and time consuming work, but you gotta do it if you want your website to look right.
Somewhere along the way, websites started getting much more complex. Some genius decided to make a big online store where you could get pretty much anything, from books to socks to computers to whatever. Now, imagine if you will, the task of hand coding every page. For every item and every view and every picture and every review there was some army of poor bastards on the other end furiously coding away to keep up with the inflow of new inventory/users/reviews. I recently read about some guy who worked for an online newspaper, and he supposedly spent 14 hours a day coding HTML for the site before he figured out that there was a better way to get your steady flow of content up there.
The answer lies in scripting language and a database.
A scripting language is basically the same thing as an interpreted language in as much as it's not compiled, or even thought of until “runtime”. At runtime, the computer gives instructions to something, probably your browser, which then passes the request along to the code. It's at that time (runtime) that the code is crunched into the binary code that your computer can actually do something with on the processor level. This is hyper-simplified.
What this means in the Amazon example is that instead of hand-coding every page, why not just build a template that gets populated with the data that's requested at the time? Then you don't have to rebuild the wheel every single time a page request is made. Well, how do you build a template? With a scripting language, of course!
I'll just cut to the bird-eye view. When you go to Amazon, and it knows that it's you, and it spits out all that stuff that you should buy based on past purchases, that's because all of that info is stored in a database. When you click on an item and go to that items page, and the page looks just like the last one, that's because all that info is stored in a database. There's a table in there for every one of you, and for every item on the site. The table for you has all of your info in it, including past purchases, addresses, billing info, etc. The table for each item has the description, pictures, reviews, links, etc.
The point is that each of these tables is filled out in a standard way with all of their pertinent data. Why this is cool is that some automated process can then do something with it, like fill out a web page, or display your customer profile. The scripting language is what automates the process. It provides the instructions for the web server on the other end of your computer to “dynamically” make up the HTML needed to display the pictures and profile info and whatever else in a cohesive way across the entire website. Dynamically means that you don't need an army of HTMLers because the computer is writing the HTML at the time of the page request. Every different kind of page on the site (say reviews vs. the home page) has it's own scripted template that's called up when the appropriate request is made. Kinda cool, no?
So, a database is somewhere between MS Excel and that big Tupperware bin where you keep all your summer clothes. We'll dive into the beauty of doing things this way later.
A few database driven websites:
Extra credit : Start paying attention to URLs, especially whenever the letters .php are used. PHP is a very widely used scripting language for making database driven websites, and if .php is included somewhere on your address bar, you're looking at data that's stored in a database.
#general-development
I went to school in Boone, North Carolina at Appalachian State University. When I picture heaven in my head, it looks more or less like what I remember the drive from Banner Elk to Boone along 105 looking like – Grandfather Mountain, the old Gold Mine tourist joint in Foscoe, Hawk's Nest ski area where I used to be a lift operator, that sandwich place on the corner of the turn to Valle Cruces, the leaves in the fall, the walk up Howard's Knob, hiking up Table Rock in the middle of the night to see the most beautiful sunrise a few hours later. I get goosebumps thinking about it.
For some reason I've been thinking about that drive a lot the past two days. I wonder what it looks like now. There used to be this rock quarry at the top of the hill as you were leaving Boone toward Foscoe, at the corner of the 105 bypass. I was cleaning up my yard this week and I remembered driving past that quarry probably 8 or 9 years ago. This was just as I was getting into DJing in college. I didn't go to Phish shows in college like all of my friends did. The String Cheese Incident was just starting to get really big about the time that I graduated. Instead I got into the whole Electronic/DJ/Rave scene in the late 90s. This was before kids started sneaking out of the house and getting themselves killed ingesting drugs meant to put Fido to sleep at the vet, in a golden year-long period where the rave scene in NC was exactly where I wanted to be. I spent my time driving to big parties all over the SE to go see DJ Icey, Dieselboy, Sasha and John Digweed, Jimmy Van M, Simply Jeff, Scott and Robbie Hardkiss, DJ Keoki, Donald Glaude, DJ Dara...
I went to this one party in my hometown of ATL one time. The headliners were Simply Jeff and DJ Icey and Dieselboy. There was a point earlier in the night where I was dancing my ass off and I noticed that there was this circle around me of people watching. It was weird because I wasn't paying any attention to anything other than the basslines coming out of the system courtesy of Mr. Icey. I still remember this one record he played that night – the vocal hook was “can you feel the BASS”, and when the record said 'bass' there was this note that came out of the subs that can't be related verbally. It shook the world. It made my hair stand up. It made the entire party, all 5000 of us, stop dancing and look around at each other. Yes, we could feel the bass. Oh my God, could we feel it...
Later in the night I found myself up on this stairway platform looking out at the whole crowd rippling like a flag blowing in the breeze. I came down the stairs and some girl walks by with a backpack full of glowsticks. I said it was the late 90s, didn't I? I guess this girl was part of what I now know as the 'ambiance crew'. I'd never seen her before, but she walks by me and pulls out a glowstick and cracks it open and hands it to me. I still have that glowstick. That glowstick was the light bulb that went on in my head at the moment that I looked around and noticed 5000 people dancing in unison to 1 guy's music. 1 guy hidden in some corner of that place had this entire party under his spell. 1 guy was responsible for guiding this entire party and making them get down like they'd never gotten down before in their lives. That's was the moment I knew that I needed to get a pair of turntables and start figuring this thing out.
This was left for us earlier today in the comments:
You may or may not know this but Apple and Stanford are offering free iPhone development courses online http://www.engadget.com/2009/04/03/apple-and-stanford-to-offer-free-iphone-development-courses-onli/
And it's true. The whole thing was available in PDF form. I hadn't even thought about it until just now (thank you Chuck), so I'm gonna go download the course again at
http://www.mobileorchard.com/take-stanfords-iphone-programming-class-for-free
I'm screwing around with the host settings of this website pretty much continually. If you should happen to come by here and find a “Site doesn't exist” or other such notice, please check back soon. I probably just changed some setting to see what would happen and ended up hosing the entire site. Thanks to Scotty Baron for the word “hosed”.
A fantastic Getting Started with Rails guide. I'll probably reference this page later.
It occurred to me last night during the show (where I do all of my best thinking) that of all of the thousands of websites that I've pored over in this quest, of all the books that I've perused, of all the tutorials that I've read and followed through, the one thing that was missing was the square one explanation of what the hell we're doing here in the first place. Everything I've read so far assumes some sort of foreknowledge or education to prepare oneself for the task of learning software development. I have no such education. A few very basic concepts have to be put into place before one can train oneself to think like a computer (or more likely, to train your computer to think like you).
As I've said before, software dev is a new thing for me, so if anyone has any corrections or additions to anything I write, please send them to me. I am now attempting to write the complete and utter morons guide to learning software development. I want to be factually correct, of course. And without further adieu..
Here's the thing. Computers don't think like you and me. They think in 1s and 0s. Every programming language ever invented has basically been a way to translate what a human wants to have done into a stream of 1s and 0s to be read by the computer. The translation into binary can happen at a couple of different points along the way from programmers head to users computer screen, but there are basically two different methods for how different languages do this. Some languages, especially the older (pre-1990) ones are “compiled”. That's a really byzantine process that basically involves taking the code that you have written, translating it into some 1s and os, linking it with other files in your code, and wrapping it up into an “executable” – a program that you can execute on your computer or whatever. The advantages of this method are thus:
Some larger applications can have bazillions of lines of code to execute (think about a Windows installer disk). If all of these lines of code were translated into binary at the time you run the application, the app would run so slowly that it'd be useless. This was obviously a big deal before CPU speeds got to be what they are now. Back in the day, the act of compiling was in itself a very time consuming procedure, and still can be today, so by doing it beforehand you save that much time and effort on the part of your computer. You also have the advantage of tuning your application on a lower level with regards to interaction with the Operating System (we'll get to that), thus making it possible for your app to perform much more nimbly. So basically, most of the benefits of compiled languages are in the performance arena.
The disadvantages of compiled languages have mostly to do with portability. Since the code has already been bundled into binary, the only machine on which the executable will run is the one for which it was written. That's why you have Mac and PC versions of the same software on the shelves. They had to be written and compiled separately since the two different computers have different methods of operating at the OS level and thus need different sets of instructions to accomplish the same task. C is the mother language from which virtually all other languages are descended, and the perfect example of a compiled language. iPhone apps are written in a language called Objective-C, descended from C with an OO twist. Since iPhone apps are written for one device and one device only, and since application performance on a phone really matters (think battery life), compilation makes perfect sense.
The other side of the coin, and I've been looking for this side for a while and have only recently found it, is an “interpreted” language. HTML is the perfect example of an interpreted language. HTML is only lines of code, just like an iPhone app or the Windows executable. The difference is that you leave the code alone and don't compile it. The code is instead “interpreted” at the time it's run, in the case of HTML by the browser that you're working in. The pros and cons of interpreted languages are exactly the opposite of compiled languages. You only write the code once and it works for every machine out there. It's left alone until that particular section of code is needed at “runtime”, at which point whatever is doing the interpreting, be it the browser or the server on the other end, crunches your code into binary and spits it out into whatever the application needs at the time. As computers have gotten more and more powerful, this method seems to be gaining a lot of traction as the preferred method of writing new applications. Some examples are JavaScript, PHP, and Ruby – languages that get chewed up into something actionable at runtime. In the case of Ruby on Rails, which is the hot-shit dev platform at the time of this writing, the code actually gets interpreted twice – once on the server side where the code is translated into HTML, and again on the browser side when the HTML is translated into something that you can read and work with.
Twitter was originally written with Ruby on Rails. If you think about it, Twitter is just an application that communicates over the internet. It really only does that one thing. They don't even display advertising (yet). Many of the traditional features of a commercial website are not there, so why write it in HTML? Especially when you have this vast database of information that gets displayed uniquely on every different computer screen out there, it's a perfect solution. Parts of Facebook are written with PHP, another interpreted language. PHP is designed to interact with info in a database – users and their profiles, pictures, text, whatever – and to write the HTML to display that info on the fly at the time of the page request.
Pretty cool, huh? Comments? Suggestions?
#general-development