Slashdot Mirror

The ratio of CEO compensation to average worker compensation is now approximately 10 times its value in 1950. This is approximately commensurate with the average increase in the Dow average adjusted for inflation.

Right; and one should hardly be surprised by this since our government continually passes more and more regulations that generally only benefits big businesses. The barrier to entry for a small or medium-sized firm to get on a public stock exchange is enormous. When competition is limited, one should not be surprised when the market can no longer efficiently remove wasteful players. Paying prices vastly more than necessary to secure a proper executive is, of course, very wasteful. But this is not a fundamental issue with CEO pay, this is an issue with regulation that keeps smaller firms out.

But why should CEOs receive the entire benefit of a growing economy when all actors have contributed to that growth? CEO compensation has no correlation with company performance.

As I see it, the problem has nothing to do with a free vs. a coerced market. The problem is that the market of executive compensation is entirely divorced from the market at large. "Stockholders... vote... for whatever the management recommends no matter how poor the management’s record of accomplishment may be". This is what I mean by oligarchy: a few privileged elites have control over this smaller market without the essential feedback cycles that stabilize prices in the larger economy.

Yes, and this smaller market is much easier to manipulate when it remains artificially small due to artificial barriers to entry. That said, your definition of oligarchy is quite arbitrary; even if you could absolutely measure the power the "privileged elites" have over a smaller market, at what ratio of power to size does it constitute an oligarchy? I do agree with your sentiment, and I think my paragraph above speaks to it.

The issue is that the market value of labor has plummeted in relation to productivity and in relation to the value of top earners. In the 50s one could work part time at a minimum wage job and pay rent and college tuition and walk away with a degree free and clear. Today, just to pay rent, one needs roommates or more than one part-time minimum-wage job, let alone any ability to pay for education in order to get a better job.

1950: $0.75/hour * 20 hours * 50 weeks = $750 wages $42 * 12 months = $504 rent $35 * 4 quarters = $140 tuition

2013: $7.25/hour * 20 hours * 50 weeks = $7250 wages $602 * 12 months = $7224 rent $3917 * 2 semesters = $7834 tuition

How do you measure productivity? GDP is a pretty useless measurement. Also, there is this silly notion that public sector consumption should actually be counted as production. Since there is no objective way to measure public sector "productivity" (since it is not part of a market), it should not be included in aggregates; also it is quite common for the public sector to be horribly inefficient with its "funds". Government makes up

The ratio of CEO compensation to average worker compensation is now approximately 10 times its value in 1950. This is approximately commensurate with the average increase in the Dow average adjusted for inflation.

But why should CEOs receive the entire benefit of a growing economy when all actors have contributed to that growth? CEO compensation has no correlation with company performance.

As I see it, the problem has nothing to do with a free vs. a coerced market. The problem is that the market of executive compensation is entirely divorced from the market at large. "Stockholders... vote... for whatever the management recommends no matter how poor the management’s record of accomplishment may be". This is what I mean by oligarchy: a few privileged elites have control over this smaller market without the essential feedback cycles that stabilize prices in the larger economy.

The issue is that the market value of labor has plummeted in relation to productivity and in relation to the value of top earners. In the 50s one could work part time at a minimum wage job and pay rent and college tuition and walk away with a degree free and clear. Today, just to pay rent, one needs roommates or more than one part-time minimum-wage job, let alone any ability to pay for education in order to get a better job.

1950:
$0.75/hour * 20 hours * 50 weeks = $750 wages
$42 * 12 months = $504 rent
$35 * 4 quarters = $140 tuition

2013:
$7.25/hour * 20 hours * 50 weeks = $7250 wages
$602 * 12 months = $7224 rent
$3917 * 2 semesters = $7834 tuition

I believe that raising the average wage will have a better impact on the economy as a whole than raising executive compensation. I believe that income inequality is a social ill that should be addressed through policy -- not by Marxian state capture of the means of production and not through Randian private hoarding of the means of production, but through a hybrid realistic approach like "all employees should receive stock options or profit sharing if executives do".

Without knowing what you're actually doing it's kind of hard to make any recommendation.

Where I work, we find Tableau to be a good middle-ground between Excel and full SQL environments. It's not really a spreadsheet perse though.

For a slightly more detailed comparison of Warmest 100 vs. Hottest 100, here's a chart a mate did which includes some commentary:

http://public.tableausoftware.com/views/WarmesttoHottest1002012/Dashboard

Right. The folks at Tableau did some visualizations with our data and you can see that most hits are written in the Key of C major: http://public.tableausoftware.com/views/billboard_songs/AnatomyofaHit?:embed=yes&:tabs=yes&:toolbar=yes However, it's likely that most songs are written in C major since it's the easiest to play on piano (all the white keys). So, yeah, that's the next step for our project.

And yet, one of my toy projects in grad school generated over a billion data points that needed analysis. The only database server I had access to choked on the first analysis pass (of about a dozen). It took about three months of processing to get to a point that could even be considered acceptable. Since then, on a whim, I've redone the program using Hadoop and HBase. A MapReduce job completed the analysis, on worse hardware, in less than a day. A major contributing factor was the lack of a rigid structure in HBase, which greatly improved the organization of data.

NoSQL solutions are certainly not the best tool for every job, but neither are normal relational databases. In my opinion, it's worthwhile to be familiar with both, and to be able to choose the right one for every task.

Or you could have just used an analytic database. There are any number of them available these days including commercial offerings like Vertica, ParAccel, VectorWise, and Greenplum, not to mention open source projects such as MonetDB and C-Store.

Hell, the darn things aren't even that hard to write - I built a commercial grade one with two other guys in about 18 months. It can query 1 billion rows in seconds and comes built into my company's data visualisation product Tableau Desktop. Free trials, and student rates. Plus, it talks to all the other databases I mentioned (well, maybe not the OSS ones, but if they have ODBC drivers, you should be able to get it hooked up).

Now, for data shaping, some of your tools are pretty useful. But for raw speed for computation and aggregation, I'd go with a tool designed for processing numbers, not text.

It's easy to generate beautiful PDFs from well-structured data but it's much harder to go the other way. Would you rather have budget figures (for example) as a CSV file in a well-defined format or as a PDF of tables and graphs?

More importantly, it's then easy to import that data for visualization and analysis purposes. Data presented as a PDF file is effectively so inaccessible that it will rarely be extracted for further analysis, meaning that some gov't functionary becomes responsible for the presentation and analysis instead of members of the public. Then a panoply of tools become available for finding out things from that data that no one ever knew were there. Something like Tableau Desktop can slurp in CSV data (or data imported to a slew of OSS or commercial DBs) and allow very rapid exploration.

As an aside, I will point out that CSV is an _evil_ format. Did you know it can be generated in localized forms (without any distinguishing metadata), that mean comma is supplanted for use as a thousands separator? Oops. Really, what idiot thought it was a good idea to have a localized data format... Much better to use a serialization format like Avro which uses a compact serialization for tabular data (akin to Protocol Buffers or Thrift) and the schema data (i.e. the description of the table's structure: columns, types, etc.) as a sidecar file in JSON.

Yes, the city of dc did a lot to democratize data for citizens while Kundra was CTO. It was part of their Capstat project. You can see some of DC's data as part of that project here: http://www.tableausoftware.com/blog/Vivek-kundra-dc-cto-take-obama-CTO-post

Ick. I used to do that kind of stuff - I was a major Excel junkie. But then I got Tableau. Way better - I mean leaps and bounds better. And it's actually fun. http://www.tableausoftware.com/

I first saw a video of Hans Rosling, who had some very unique ways of visualizing data that would otherwise be useless to a simple mind such as mine.

After I watched that, I found a piece of software called Tableau. I downloaded the trial version, and really liked how easy it made visualizing data for me. I can take the data I have, and Tableau will see how it's connected and allow you to generate visual reports of the data. I'm not saying that it'll work for everything, but it certainly does what I need it to extremely well, especially for my business intelligence initiatives.

I first saw a video of Hans Rosling, who had some very unique ways of visualizing data that would otherwise be useless to a simple mind such as mine.

After I watched that, I found a piece of software called Tableau. I downloaded the trial version, and really liked how easy it made visualizing data for me. I can take the data I have, and Tableau will see how it's connected and allow you to generate visual reports of the data. I'm not saying that it'll work for everything, but it certainly does what I need it to extremely well, especially for my business intelligence initiatives.

Tableau Desktop is an interactive analysis and visualization product that connects to relational and cube data sources to help people see and understand their data. There was a webinar (slides - PDF) back in November 2008 covering Blastrac Global's success in using Tableau with their ERP system.

Disclaimer: I work at Tableau Software, so I encourage you to see for yourself with a free trial: http://www.tableausoftware.com/products/tour

Tableau Desktop is an interactive analysis and visualization product that connects to relational and cube data sources to help people see and understand their data. There was a webinar (slides - PDF) back in November 2008 covering Blastrac Global's success in using Tableau with their ERP system.

Disclaimer: I work at Tableau Software, so I encourage you to see for yourself with a free trial: http://www.tableausoftware.com/products/tour

Tableau Desktop is an interactive analysis and visualization product that connects to relational and cube data sources to help people see and understand their data. There was a webinar (slides - PDF) back in November 2008 covering Blastrac Global's success in using Tableau with their ERP system.

Disclaimer: I work at Tableau Software, so I encourage you to see for yourself with a free trial: http://www.tableausoftware.com/products/tour

• Many people are much more proficient in Excel than in any other software, due to familiarity. The cost of moving the data over to other software and then figuring out how to do what you want to do is often not worth the time required.
  
• Spreadsheets are inherently more flexible than a database. This is both a strength and weakness, but there are plenty of times when you need to do some sort of funky calculation to the data that is much, much easier to implement in a spreadsheet than via SQL (calculating a moving average of X records comes to mind).
  
• I would argue that a DB is often not the best choice, given the difficulty of implementing certain calculations in SQL (yes, I know about Codd and relational theory, but just because something can be done in an RDBMS does not mean it can be done easily). Often, something like S+/R, Matlab/Octave, SPSS, Tableau, etc., is the better choice- they generally allow you to perform manipulations as flexibly as in Excel, with better performance and less likelihood of errors, but more easily than doing everything in SQL. But many people aren't familiar with these. That doesn't mean they shouldn't be, but they aren't.
  

http://www.tableausoftware.com/

Sure, it's not open source and it costs money, but it does everything you're asking. For you to roll your own, your company is going to end up paying you a lot more than that costs and then what happens when something doesn't work or if you leave?

http://www.stanford.edu/class/ee380/Abstracts/0605 03.html
http://www.stanford.edu/class/ee380/schedule.html

Tableau doesn't say who are its customers, [...]

Look at their brochure: one samble uses a Starbucks cube.

And on the customers page, the world's largest Internet search provider is quite obvious.