I routinely work with CVS files that are exported from the database. Since CVS files are text files, I can use this DOS command to merge multiple CVS files into a single 70MB+ CVS file.
copy *.cvs everything.cvs
With a single CVS file, I can open it in Excel and do my daily work.
I never went to community college but I'm pretty sure a database is the best place for big data.
Big data wasn't taught when I went to community college over a decade ago. I did take a course in database management for web developers. Most users today don't have direct access to the database and need to use CSV files instead.
I can't even find a text editor that will load a multi gigabit sql file.
Have you tried UltraEdit? It supports editing 4GB+ files from disk with minimum RAM usage.
So you're whining about two positions, his daughter and son-in-law, that could be questionable out of five million? It's really hard to take you people seriously.
Congress passed an anti-nepotism law after JFK appointed his brother to attorney general to prevent future administrations from appointing family members to positions in the government. Just like most reform laws from the last 50 years, Trump is ignoring that one too.
Mars' distance to Earth is closest about every 2 years. For the sake of practicality, launches only occur with a short time window unlike a Moon - Earth launch which is about every 30 days.
Or do a gravity assist with a Venus flyby to Mars.
by 2030 it could be Malia Obama getting the credit for it for all we know, and there are at least three reasons why Trump won't risk the credit going to a Democrat woman of color.
We're more likely to see Kamala Harris run for POTUS than Malia Obama.
I've been using 3GHz processors for 10+ years (dual-, quad- and eight-core). The only time I ever pegged out cores at 100% is from encoding videos. Everything else uses one or two cores at most. We got enough CPU to handle everyday applications.
It's almost as unlikely as him ever getting elected President.
Trump's electoral victory was probably a statistical fluke. Maybe Biden will have better luck than Hillary in 2020, assuming that Kasich doesn't knock Trump out in the primaries.
Seeing how things are going, he'll likely get a 4th term, or more.
Congress will have to vote to repeal or modify the 22nd Amendment, which is unlikely as the Republicans can't negotiate among themselves and gave the Democrats everything they wanted in the budget deal to avoid a government shutdown.
Trump was probably flipping TV channels when he came across a documentary about Kennedy's speech for putting a man on the moon by the end of the decade. What worked for Kennedy should work for Trump, right?
What made Kennedy a "great" president was the fact that he got assassinated. If had lived, he might have washed out like the future Kennedy in Red Dwarf who goes back in time to assassinate himself.
That Trump will finish a first term much less get reelected to a second term is as unlikely as NASA to send astronauts to Mars in the next eight years.
Have you ever found anybody who cares about your life story?
Yes. I'm known as the guy with the funniest stories at work.
Or do you just keep posting it because it's either that or cutting and you're all out of razor blades?
I'm living out my favorite demotivator poster that hangs in my home office: "Mistakes:
It could be that the purpose of your life is only to serve as a warning to others."
How many women have you made love to during that same period of time? Please break it down by real women, women you've just fantasized about, women who had penises, and crevices within upholstered furniture.
Whatever James Bond did, I did it twice as much.;)
Be sure to include a chapter on how you went to community college. That's the sort of unique experience that most of the rest of us here have never experienced, since we went to a real college (or university, for those Brits among us).
That might be a separate book unto itself. I spent eight years in Special Ed classes, skipped high school, went to community college for four years (two years of remedial coursework and two years for General Education), transferred to the university, got kicked out of the university, and went back to school a decade later to get my A.S. degree in Computer Programming.
So your CSV 'database' is woefully incomplete, with it missing almost a decade of comments.
My "database" is incomplete because Slashdot no longer has my earlier comments. Some kind of housekeeping or accident may have deleted my earlier comments.
Yet in another comment here today you wrote, 'I ran a search for "dead wood" in Excel and a few seconds later found nothing.' Has it occurred to you that the comment you say you can't find might exist within the years you do not have data for?
That was in reference to earlier discussion that took place in recent months. The words "dead wood" or "deadwood" aren't words that I normally use, especially in reference to my government IT job.
No he is taking the piss because CSV is probably one of the worst ways to store verbose text data, as all escaping sequences are likely to turn up in the text, plus newline characters complicate record delimiting.
Wrong thread then. I used a text editor to fix HTML back in 1997.
Out of interest, how are you handling this?
Python and Beuatiful Soup 4 to parse the HTML tree. Once I have the comment text, I use string manipulation to remove the newlines, extraneous tags and white space. The end result is straight HTML.
<div class="quote"><p>Did you export the data into to CSV format and then import into Microsoft Excel?</p></div><p>Post to the wrong thread, friend?</p>
He's scraping Slashdot to help him debate people on his post history...so at the very least, clinically insane.
That's one purpose. I've wrote quite a few stories on Slashdot over the years, most I remembered and some I have forgotten. I plan to write a series of essays on my misadventures in Silicon Valley.
Slashdot Mirror
CVS files for big data??
I routinely work with CVS files that are exported from the database. Since CVS files are text files, I can use this DOS command to merge multiple CVS files into a single 70MB+ CVS file.
With a single CVS file, I can open it in Excel and do my daily work.
I never went to community college but I'm pretty sure a database is the best place for big data.
Big data wasn't taught when I went to community college over a decade ago. I did take a course in database management for web developers. Most users today don't have direct access to the database and need to use CSV files instead.
I can't even find a text editor that will load a multi gigabit sql file.
Have you tried UltraEdit? It supports editing 4GB+ files from disk with minimum RAM usage.
https://www.ultraedit.com/
Can I quote you on this in about four years when he gets re-elected?
Sure. Just remember that futurists don't predict the future.
https://www.wsj.com/articles/think-like-a-futurist-to-be-prepared-for-the-totally-unexpected-1483272006
So you're whining about two positions, his daughter and son-in-law, that could be questionable out of five million? It's really hard to take you people seriously.
Congress passed an anti-nepotism law after JFK appointed his brother to attorney general to prevent future administrations from appointing family members to positions in the government. Just like most reform laws from the last 50 years, Trump is ignoring that one too.
http://time.com/4574971/donald-trump-transition-jared-kushner-legal-anti-nepotism-law/
It's really hard to take you people seriously.
If Hillary was POTUS and she appointed Chelsea as a special adviser, the Republicans would be screaming for impeachment.
Mars' distance to Earth is closest about every 2 years. For the sake of practicality, launches only occur with a short time window unlike a Moon - Earth launch which is about every 30 days.
Or do a gravity assist with a Venus flyby to Mars.
https://en.wikipedia.org/wiki/Manned_Venus_Flyby
by 2030 it could be Malia Obama getting the credit for it for all we know, and there are at least three reasons why Trump won't risk the credit going to a Democrat woman of color.
We're more likely to see Kamala Harris run for POTUS than Malia Obama.
https://en.wikipedia.org/wiki/Kamala_Harris
I've been using 3GHz processors for 10+ years (dual-, quad- and eight-core). The only time I ever pegged out cores at 100% is from encoding videos. Everything else uses one or two cores at most. We got enough CPU to handle everyday applications.
Not really. The POTUS swings between the parties a lot.
An electoral victory without the popular vote has happened only five times. Twice in the 21st century and three times in the 19th century.
http://www.factcheck.org/2008/03/presidents-winning-without-popular-vote/
Post on Slashdot. What else would any wannabe programmer would do?
As a sometimes-Republican with a folder full of Creepy Uncle Joe memes, I beg you, Please, please, nominate Biden.
Forget about the Uncle Joe memes. The Onion articles will be hysterical.
http://theweek.com/articles/468552/6-best-onion-parodies-joe-biden
Now go - write a devices driver or Linux kernel in Python.
I wonder if that can be done. Write the driver in Python and use Cython to convert Python to C.
I once wrote a Python script to roll a pair of dice 1M times that took 123 seconds. Used Cython to convert the script to C that ran in a second.
It's almost as unlikely as him ever getting elected President.
Trump's electoral victory was probably a statistical fluke. Maybe Biden will have better luck than Hillary in 2020, assuming that Kasich doesn't knock Trump out in the primaries.
Seeing how things are going, he'll likely get a 4th term, or more.
Congress will have to vote to repeal or modify the 22nd Amendment, which is unlikely as the Republicans can't negotiate among themselves and gave the Democrats everything they wanted in the budget deal to avoid a government shutdown.
You underestimate the stupidity of humanity. Have you ever met people?
I work in IT support. Are people as bad as users?
Trump was probably flipping TV channels when he came across a documentary about Kennedy's speech for putting a man on the moon by the end of the decade. What worked for Kennedy should work for Trump, right?
What made Kennedy a "great" president was the fact that he got assassinated. If had lived, he might have washed out like the future Kennedy in Red Dwarf who goes back in time to assassinate himself.
That Trump will finish a first term much less get reelected to a second term is as unlikely as NASA to send astronauts to Mars in the next eight years.
how are these on any way like python or some other programming language?
Spaghetti code.
Have you ever found anybody who cares about your life story?
Yes. I'm known as the guy with the funniest stories at work.
Or do you just keep posting it because it's either that or cutting and you're all out of razor blades?
I'm living out my favorite demotivator poster that hangs in my home office: "Mistakes: It could be that the purpose of your life is only to serve as a warning to others."
I'm sure Slashdot keeps a record of my comments for you to confirm my rantings on this.
According to my scraping script, you're a noob on Slashdot. Soapbox much?
How many women have you made love to during that same period of time? Please break it down by real women, women you've just fantasized about, women who had penises, and crevices within upholstered furniture.
Whatever James Bond did, I did it twice as much. ;)
Be sure to include a chapter on how you went to community college. That's the sort of unique experience that most of the rest of us here have never experienced, since we went to a real college (or university, for those Brits among us).
That might be a separate book unto itself. I spent eight years in Special Ed classes, skipped high school, went to community college for four years (two years of remedial coursework and two years for General Education), transferred to the university, got kicked out of the university, and went back to school a decade later to get my A.S. degree in Computer Programming.
So your CSV 'database' is woefully incomplete, with it missing almost a decade of comments.
My "database" is incomplete because Slashdot no longer has my earlier comments. Some kind of housekeeping or accident may have deleted my earlier comments.
Yet in another comment here today you wrote, 'I ran a search for "dead wood" in Excel and a few seconds later found nothing.' Has it occurred to you that the comment you say you can't find might exist within the years you do not have data for?
That was in reference to earlier discussion that took place in recent months. The words "dead wood" or "deadwood" aren't words that I normally use, especially in reference to my government IT job.
No he is taking the piss because CSV is probably one of the worst ways to store verbose text data, as all escaping sequences are likely to turn up in the text, plus newline characters complicate record delimiting.
Wrong thread then. I used a text editor to fix HTML back in 1997.
Out of interest, how are you handling this?
Python and Beuatiful Soup 4 to parse the HTML tree. Once I have the comment text, I use string manipulation to remove the newlines, extraneous tags and white space. The end result is straight HTML.
This is what the previous comment looks like in the CSV file:
You need to get out more precious.
My comment history goes back to 2008. Not sure what happened to the comments from 1999 to 2007.
I think you strongly underestimate the data and metadata available on the net.
Here's the metadata for my comment history:
He's scraping Slashdot to help him debate people on his post history...so at the very least, clinically insane.
That's one purpose. I've wrote quite a few stories on Slashdot over the years, most I remembered and some I have forgotten. I plan to write a series of essays on my misadventures in Silicon Valley.