I don't know if most/. people believe what you think they believe. I am against all income taxes, all payroll taxes, all taxes that involve money making and money movement as opposed to money spending.
I see it as a virtue when anybody gets away with not paying any of the above mentioned taxes and I see the state as a thief, trying to steal money.
Also I truly believe that it is best for economy when money that is not spent but invested must not be taxed and people and businesses must not be regulated in any way, but also never subsidized in any way.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Yeah, I have one application working that way, it works, my basic question around here was whether Tomcat is doing anything about it or not.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Yeah, let me post something here right now. This is a list of requests from today (the day is not over yet, it's not US timezone, so don't be surprised.) I just cut that out of the log table, I am not showing the incoming IP addresses, user names, types of the reports that are executed, the parameters passed into the reports, etc.etc. Just times and milliseconds.
You think people, who are busy working, care about your idea of queues? You think it doesn't let the people to 'feel in control' in a system that does in seconds and in some cases in minutes, what they are used to be taking tens of minutes and hours, and in some cases not being able even to get that kind of data, because the solution is much bigger than just a silly small part that generates reports?
I feel the arrogance in this place, oozing through the pores. The people in my systems, they are working, not playing, they don't care about 'feeling in control', they care about getting stuff done. Many of them are running reports one after the other, after the other just because they adjust some very small filter detail and need to see the result right away. Giving them one more thing to do - some job control queue, yeah, that'll make them happy.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Excuse me for the less than stellar writing here, but I make mistakes even when I type in my own language, so it's not surprise that I sometimes mistype homophones.
I bring shame into it because you obviously get snippy when anyone comes close to hitting a nerve.
- the place is full of geniuses here, everybody is happy to propose a total rewrite for something that works, citing their 'tens of years of experience' as a 'sound reason' for that idiocy.
If you've got the architecture of last resort or whatever and your clients are just so thankful they have what they have, good for you.
- nice jab, totally unfounded in reality, just like the rest of the comments here.
I've encountered plenty of examples of people like yourself in my travels and a lot of 'em can just ride that wave for years until it crashes. Hang ten.
- I'd be surprised if you found many 'people like me', you have no clue, again.
And I do know your business. Reporting, db's, Java, enterprise apps? It's like, uhm, I've been doing that for 12 years across 5 different industry categories for the consulting firm I work for. Ha, I'm dealing with retail right now! And, I've done POS and ecommerce for retail clients in the past.
- yes, and I have worked in more categories than that as a contractor, but have you ever started your own business, created all of your own software from scratch? For ONE category ever? All business software in about one and a half years, including SCM that ties together stores with office with suppliers with manufacturers? Have you had to sell your solution all by yourself to more than one supplier, more than one manufacturer? Yeah.
If I produced a system that had performance characteristics like what you're describing, our clients would terminate our contracts.
- another snark remark. You have no idea what you are talking about. You have no idea how many systems are tied together to produce a solution. But you are willing to continue display your supposed 'superiority' regardless of the total ignorance about the counter-party. As I said, I love/.
Before consulting, I was doing C++ and assembly systems-level stuff for another 6 years. So I know what brute force looks like, and I know what finesse looks like. Don't get cocky and think you have anything special or revolutionary there, there's no secret sauce anymore.
- I am happy for you, but not surprised. All this experience, but nothing useful to say.
Again, I see a lot of people here, proposing the same thing: -you have it all wrong, you need a rewrite, you need this, you need that. They are all bringing up their 'experience', totally oblivious about the counter-party, totally irrelevant to the question, sounding exactly as you just expressed it - cocky, and nothing useful.
No surprise so the people in this field have such a poor rap.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
wow, another genius. So I am waiting for the email.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Money well spent when I spend it on a vacation, everything else is a waste.
As to hiring a consultant - for what purpose? To solve a problem that happens 1-2 times a month on a system that is being used by 50 people 12 hours a day? I already have a solution, and it doesn't involve buying into nonsense.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
ha ha ha, well, anything to skip looking at your pathetic faces.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
We understand the question, but the answer is your architecture is poor.
- are you providing money to buy, maintain, support more hardware and software?
your job is to make an architecture that works
- I explained in that link. It works.
and flfills the user's request.
- and I explained in that link, it fulfills them.
My batch-middleware suggestion (which to your clients could be made to look the same as real-time interactive with a job-status page), could even be implemented on the same server
- an irrelevant suggestion. There is no difference between a middleware solution and this one, when in fact all this solution does is generating reports, that's its purpose. All of the resources of the hardware are allocated to it. Every day of the month, 12 hours a day, 50 people use it to generate reports, that's all it's used for. 1-2 times a month (once in January and once in December at this point, from memory), the thing runs out of memory and CPU due to too many abandoned threads. As I explained in that link, near all requests finish under 7 minutes. Very few finish under 10, but that's the maximum, we shouldn't allow requests to run past 15, nobody is waiting past 10 anyway.
There is nothing that middleware would provide, except slowing everybody down in 99% of cases, because it would rely on queues, it would synchronize the work, most of which doesn't need to be synchronized. The problem presented? Stopping some abandoned work threads from going past 15 minutes.
The solution given on this site? Rewrite everything. The reason? apparently 37 years of experience. Too bad none of it is in business.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Beautifully done, I love/. an entire number of assumptions, not backed by any reality, but made with complete certainty. It's an amazing site and sight.
1. I am no longer looking for a Tomcat provided solution, I am quite certain all of the changes must be in application. 2. Would you like to pay for the new tool? The transitioning to the new tool? Maintenance? Support? Training? 3. Are you sure that your tool would do a better job than what is currently done with that same data, regardless of how the data is structured? Good to be so sure. I wouldn't be so sure, but that's because I know what the data is, which parts of data are cached, what the structure is. Also, I have this tingling sensation in my left foot, can you tell me if a white pill would do more good for that than this green pill? 4. You are sure that you know everything about the requirements here, so your educated guess about my platform must be right, obviously. 5. I certainly know that I am not going to find any good answers on this forum, but that's not going to stop me from posting. I have an hour to waste. 6. I was there since java 1.0:) working merrily on an AT&T-Canada project to process large files supplied by an old mainframe that had to be shipped to rebillers, I still remember the first time I had to use java, yes. But thank you.
7. I am sure that you have many many more wonderful suggestions and they are about to come.
The short answer is that a thread can never be both arbitrarily and safely stopped. That's why Thread.stop() has been deprecated since nearly its introduction in Java.
- yes, I have been working with Java since 1997, I have already discovered this, thank you, you know.
Yes, in my case the threads are not blocked, they are doing whatever work they must, running hundreds, to thousands of SQL requests, yes I absolutely can have them implement their own way of timing out because of this specific way they are doing work.
Yes, my question was about whether Tomcat is doing anything about this, providing any interfaces/interrupting listeners for it, or whether this has to be done by the application on its own. So you believe that Tomcat is not doing anything in particular to address this, I couldn't find anything in Tomcat that did it so far, so I am beginning to believe that's how it is.
Thanks again.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Yeah, well, you took the bait and fed the trolls with all the cussing.
- wasting some time, no question about it.
If an in-memory cache is too hard to work out, then just consider a "job" metaphor.
- I have 15G of RAM, 50 users. All product definitions are cashed, but cashing all receipts from all stores and years, etc. and still to allow the container to have enough space to generate all the reports.... I'll need much more space than that, an order of magnitude more space than that.
Whether it's your users resubmitting or a refreshing, doesn't matter: you're making your users wait too long and they think their connection has stalled so that's why they're trying again.
- the users know what amount of time their reports take normally. They know which reports take a few seconds, they know that sometimes they need to wait a few minutes, that's just normal for them, they've been using the system for a year.
In our case 70% of all requests come back in under 10 seconds, 25% last between 10 and 120 seconds, the last 4.99% take up to 7 minutes and then there are odd cases, when users set up a report that normally would return in about half an hour. Our system is used to generate very large reports, in fact before this system the users could wait for their reports for many hours, sometimes days, so they are spoiled now, but not 'Google' spoiled. Saying that "I should never" make a user wait for more than 20 seconds:) Have you ever dealt with retail?
But, if two or three concurrent requests can clog/kill your server, you're doing something wrong.
- as I said, we have 50 people working in various geographical locations and this problem comes up 1-2 times a month, a very specific set of circumstances.
Also, training your users that your webapp just "takes a long time" isn't a very effective approach - grumbly users will turn on you eventually. It's your job to keep users happy and keep your reputation spot-free.
- thank you for telling me my job, but you don't know my job:) I love/.
I agree with the other poster, perhaps you need to invest in a BI solution. Or, take the graph generation process offline with queues and email if BI's out of your range. What you are describing does not sound workable; trust me, it's rethink time, and there's no shame in that because you're making things better.
- I don't know why you are talking about shame, I have no reason to get anything new to solve this problem. It would be very silly and ineffective to buy more hardware and to transition all of the reports from one system to another, when in fact the entire system is created just to generate the reports.
The only issue that exists in this system, which works fine for the 50 users we have every day of the month, 12 hours a day, is that 1 or 2 times a month too many reports are created at once that are dumped and re-created again by enough people that the VM itself starts running low on memory and CPU, and this can be easily fixed by killing a few threads that go overtime.
Imagine you own a piece of equipment that does everything you need, much better than a piece of equipment you used to have for years, it's orders of magnitude faster, it's actually distributed geographically while the old systems weren't, it is really fast compared to previous systems, yet once in a while the system goes off-line simply due to too many requests that are not done yet, but that are really just run abandoned. Imagine that you are not 'Google', you are not going to be 'Google' either. It's a normal situation.
To come to a business and say: because of this specific issue the entire system needs to be recreated in some other way, rewritten, new hardware needs to be bought, supported, etc. That's all great, but the question is simple: why? It's a silly suggestion, no offense, but it's obviously made by somebody not running a business.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 0
You can go ahead, asswipe, use your tongue.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Yeah, I don't care about/. AC crowd, they are clowns.
As to the issue at hand, it's not about the 'refresh' button, it is just people abandoning one report to try and generate another and it's not a normal occurrence, not something that happens all the time, but 1-2 times a month, hard to spot and slows everything to a halt. Normally it happens when the users start generating yearly graphs with too many data types on them, like sales for all products within a number of brands on a comparative graph over a period of 2-3 years for around ten stores at the same time.
So even a graph like that only takes maybe 10 minutes to create, but if they decide they want to change the parameters of the graph and abandon the first one to start generating a second, a third, and we have up to 50 users at the same time, all of a sudden there is a spike that's not normal for the app, and the tasks are still all going at the same time and the system runs out of memory and starts thrashing threads it looks like.
Unfortunately cashing responses is not what I can, since every report can be generated with tens of parameters, for arbitrary sets of stores, for arbitrary periods of time, etc. Also they are aggregate in many cases, what I means is that sales are added together in the DB based on filters, so again, I can't pre-generate discrete pieces of it to just join them together for a request.
So since this happens only on rare occasions I was thinking about timing out threads that are taking too long, that's an easy cop out, maybe just time out any task that goes on for more than 15 minutes, something like that.
I used to do that earlier - the main request thread creates a worker thread and is put to sleep for a short period of time, it wakes up, checks if the worker is done, if it's not and if it's not the timeout yet, it goes to sleep again, if it's past time out, it kills the worker thread and returns with application timeout error. It works, but too bad Tomcat doesn't allow for that itself.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 0
Another moron with a keyboard. Polling connection on what, you, nimrod? From the client it's useless, the thread is already running in the container, nothing can stop it until it's done with its task. From the thread itself - the client can still be there just fine, working on some other report. Go bang your stupid fucking head against the wall, maybe you'll set your brains straight.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
First: it's a normalized database.
Second: nobody can answer a simple question, is it possible to time out a running task through Executor thread. Your solution is not doing much good, is it? Since the user can have multiple reports running at once legitimately.
Third: I don't need your implementation, since you don't understand the question.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
excellent, all of ACs can go fuck themselves.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 0
Didn't I tell you to go fuck yourself in enough ways? Can't you take a hint?
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
If you are the original asshole (all of you, ACs, look the same to me,) then go fuck yourself. You are the one who was being the ass.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Not good. The report interface is dynamic, allows the users to change filters and ordering in real time, the entire point of this app is to do this in real time. The only problem is they are not limited on what they can do in terms of the total size of the data set. Normally even the largest requests return within 2 minutes, it's actually good enough for them.
However in cases of graphs, they can choose too many data types to be on them. For example they can choose not just brands to be on the graph, but different products under brands to be all displayed all at once for a very large time period. It's not an unusual thing and they want the result in front of them once it's compiled. Even these things return quickly enough (under 7 minutes normally). But the problem appears sometimes when they start a graph, don't wait for it to finish, change some parameters and restart the graph, etc.
The server has 16G of ram, 15G are allocated to Tomcat, so there is no memory left to do these graphs somewhere else, they actually take a lot of space as well.
I am constrained on the total memory, on CPU, on total time, basically on everything, those are hard constraints, it's not like I have a bunch of servers standing there, only one. So no middleware, nowhere even to run it, and the graph is not good if it comes too late.
Again, the question is much simpler: is there a way to time out Executor threads in Tomcat or not?
If not, then that's another reality and I have to work around it. It is possible to work around it - it's possible to stop execution of a task in the middle between SQL requests. It's possible to have do what I did sometime earlier - start an asynchronous thread for each web request, put the current thread to sleep for a small amount of time, once the time passes and first thread wakes up, check if the worker thread is done.
This way I have the time out that I can enforce - I can kill the worker thread I started.
My question is about Tomcat Executor threads - can they be killed from the app? Can they be configured to time out?
I think my question is very simple, it doesn't require anything beyond those very simple specific answers, doesn't look like people understand the questions, everybody is trying to re-architecture everything or to be a smart ass (as the rest of the thread clearly shows.)
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
ha ha ha ha, what a dumb ass, still not getting the question yet offering a useless, clueless comment here.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
Yet with your 12 years you still didn't get the question, did you? How do you time out an executor thread that's no longer useful, if it's still doing work, type of work that takes long time to do? - so if this looks like an idiotic question to you, then you haven't had to do much in those 12 years.
Secondly, it's not 'refreshes' that cause this, it's dumped requests, that are still processing. Sometimes many dumped requests.
Clearly there can be more done, more memory, a cluster, etc. Most of it would be a waste because this particular issue is only with threads that are still going, sometimes going for 45 minutes, taking up processor and memory.
So in reality, if this is not though through by the Apache team, then the question is: have they ever had to write code to produce very large reports on very large data sets and if not, why not? I am sure this is not the first time somebody is doing it in real time.
The 45 minute requests are a rarity, all of the parameters are user set, so if a user chooses a very large data set to go through and then decides not to wait for it, or tries to change the parameters of the request and restart it again - that's a normal use case.
Yes, some reports take thousands of SQL executions and massive amounts of data to be held in memory. No, this is not an every day occurrence. Yes, there are more than 1 user, around 50 people doing various things. Yes, it would be nice to have a cluster, but most of the time this is not an issue so it's not really a hardware problem.
It is possible to have checks in the code itself, and verify between executions of SQL statements whether this request should be terminated, so that's one way of doing it. But if the servlet container is not helping by providing at the very minimum a time out mechanism for the Executor threads then why not? It should be.
--
So in short, with your 12 years of experience, you are the one who looks like a dumb ass to me if you can't imagine a situation where Executor thread timeouts could be used for good before ideas like clustering and distributed nodes are considered.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 1
ha ha ha ha, idiot, didn't I say: what is the way to time out an executor thread? What is the difficult thing to understand? You can sit there and yap and yelp about code as much as you want, but what is the way to kill a thread that is taking too long? How the fuck, do you, idiot, know what a thread is doing? It's creating a gigantic 3 year based report by running SQLs against a dabase and putting together a picture that's going to be placed an HTML page.
Clearly the web is screwed up so there is no state in HTTP and the server can't easily know that the client is no longer waiting for the response, but there has to be a way to deal with requests that do not need to be executed any longer.
So, idiot, again, yap away, or provide a solution based on Tomcat configuration, otherwise there will have to be plenty of code written.
Come on, do something USEFUL once.
Re:Tomcat is as rock solid as it gets
on
Tomcat 7 Finalized
·
· Score: 0
yeah, yeah, asshole. You have any idea how to time out an executor thread after an amount of time has passed, or no? If not, shut your yap.
Slashdot Mirror
this entire article is a huge flamebait for me... must fight the urge to post lots and lots of comments on hating gov't...
I do not believe anybody should be getting any services from the gov't safe for minimum military.
I don't know if most /. people believe what you think they believe. I am against all income taxes, all payroll taxes, all taxes that involve money making and money movement as opposed to money spending.
I see it as a virtue when anybody gets away with not paying any of the above mentioned taxes and I see the state as a thief, trying to steal money.
Also I truly believe that it is best for economy when money that is not spent but invested must not be taxed and people and businesses must not be regulated in any way, but also never subsidized in any way.
Yeah, I have one application working that way, it works, my basic question around here was whether Tomcat is doing anything about it or not.
Yeah, let me post something here right now. This is a list of requests from today (the day is not over yet, it's not US timezone, so don't be surprised.)
I just cut that out of the log table, I am not showing the incoming IP addresses, user names, types of the reports that are executed, the parameters passed into the reports, etc.etc. Just times and milliseconds.
You think people, who are busy working, care about your idea of queues? You think it doesn't let the people to 'feel in control' in a system that does in seconds and in some cases in minutes, what they are used to be taking tens of minutes and hours, and in some cases not being able even to get that kind of data, because the solution is much bigger than just a silly small part that generates reports?
I feel the arrogance in this place, oozing through the pores. The people in my systems, they are working, not playing, they don't care about 'feeling in control', they care about getting stuff done. Many of them are running reports one after the other, after the other just because they adjust some very small filter detail
and need to see the result right away. Giving them one more thing to do - some job control queue, yeah, that'll make them happy.
17:44-1805 13:29-3 16:15-1536 10:32-4 17:09-11363 11:52-7703 14:38-1046 9:49-13427
17:43-856 13:28-13005 16:15-2141 10:32-17384 17:09-10 11:46-3127 14:38-11 9:49-13258
17:43-15421 13:28-13 16:15-5342 10:31-11 17:09-10922 11:46-3254 14:38-5 9:48-10
17:43-13283 13:28-2 16:15-1473 10:31-2 17:09-10 11:45-1662 14:38-779 9:48-4
17:43-13183 13:28-638 16:15-1476 10:31-10 17:08-11330 11:45-1716 14:38-775 9:48-16484
17:43-11 13:28-43 16:15-1541 10:31-3 17:08-10 11:44-7231 14:38-777 9:48-11
17:43-859 13:28-4 16:14-1541 10:30-866 17:07-11380 11:44-3501 14:38-1812 9:48-3
17:42-13182 13:27-58 16:14-1541 10:30-11 17:07-11 11:44-6752 14:38-6 9:47-10
17:42-11236 13:26-705 16:14-1541 10:30-3912 17:06-10909 11:41-4823 14:38-815 9:47-1668
17:42-11 13:26-71 16:14-1542 10:30-6 17:06-10 11:41-3155 14:37-12 9:47-4
17:42-795 13:25-3 16:13-1539 10:27-255 17:06-2 11:41-3178 14:37-6 9:47-11
17:42-13225 13:25-3 16:12-1542 10:27-6488 17:06-6 11:39-751 14:37-771 9:47-2813
17:42-11991 13:25-13804 16:12-1541 10:27-1401 17:06-11939 11:39-824 14:37-14232 9:47-3
17:42-13128 13:24-11 16:11-1542 10:27-844 17:06-2 11:37-3 14:37-12 9:46-10
17:42-10 13:24-5650 16:10-1542 10:27-562 17:06-10 11:37-4987 14:36-520 9:46-3
17:41-4581 13:24-4 16:10-1539 10:27-530 17:06-2 11:36-20852 14:36-4 9:46-7011
17:41-11378 13:23-10 16:09-1540 10:26-522 17:05-11367 11:36-11 14:36-1676 9:46-828
17:41-13104 13:23-3 16:09-1437 10:26-18443 17:05-11 11:35-4 14:35-807 9:46-10
17:41-10 13:22-47396 16:09-1438 10:26-1 17:04-10 11:35-2 14:35-11687 9:45-4
17:41-14738 13:21-2073 16:08-1437 10:26-10 17:04-11484 11:35-2 14:35-5 9:45-1554
17:41-12266 13:20-3 16:08-1434 10:26-2 17:04-10 11:35-2 14:35-4 9:45-2909
17:41-12834 13:20-3 16:08-2093 10:26-4 17:04-2 11:35-2 14:35-10 9:45-946
17:40-10 13:20-4 16:08-1435 10:26-4109 17:04-11057 11:34-2 14:34-795 9:45-10
17:40-848 13:20-3 16:07-1439 10:25-12 17:04-11 11:34-2 14:34-10 9:45-3
17:40-1636 13:20-3 16:07-1472 10:25-1395 17:03-5 11:33-48 14:34-4 9:45-5088
17:40-19147 13:20-2 16:06-1467 10:25-516 17:03-11655 11:33-11 14:33-799 9:45-7393
17:40-12187 13:20-3 16:06-1469 10:25-4 17:03-10 11:33-29 14:33-12729 9:44-776
17:40-11 13:20-4 16:06-1469 10:25-4604 17:02-11378 11:33-4 14:32-11 9:44-10
17:40-1464 13:20-3 16:06-1511 10:25-867 17:02-3 11:32-4 14:32-3 9:44-5
17:40-
Excuse me for the less than stellar writing here, but I make mistakes even when I type in my own language, so it's not surprise that I sometimes mistype homophones.
I bring shame into it because you obviously get snippy when anyone comes close to hitting a nerve.
- the place is full of geniuses here, everybody is happy to propose a total rewrite for something that works, citing their 'tens of years of experience' as a 'sound reason' for that idiocy.
If you've got the architecture of last resort or whatever and your clients are just so thankful they have what they have, good for you.
- nice jab, totally unfounded in reality, just like the rest of the comments here.
I've encountered plenty of examples of people like yourself in my travels and a lot of 'em can just ride that wave for years until it crashes. Hang ten.
- I'd be surprised if you found many 'people like me', you have no clue, again.
And I do know your business. Reporting, db's, Java, enterprise apps? It's like, uhm, I've been doing that for 12 years across 5 different industry categories for the consulting firm I work for. Ha, I'm dealing with retail right now! And, I've done POS and ecommerce for retail clients in the past.
- yes, and I have worked in more categories than that as a contractor, but have you ever started your own business, created all of your own software from scratch? For ONE category ever? All business software in about one and a half years, including SCM that ties together stores with office with suppliers with manufacturers? Have you had to sell your solution all by yourself to more than one supplier, more than one manufacturer? Yeah.
If I produced a system that had performance characteristics like what you're describing, our clients would terminate our contracts.
- another snark remark. You have no idea what you are talking about. You have no idea how many systems are tied together to produce a solution. But you are willing to continue display your supposed 'superiority' regardless of the total ignorance about the counter-party. As I said, I love /.
Before consulting, I was doing C++ and assembly systems-level stuff for another 6 years. So I know what brute force looks like, and I know what finesse looks like. Don't get cocky and think you have anything special or revolutionary there, there's no secret sauce anymore.
- I am happy for you, but not surprised. All this experience, but nothing useful to say.
Again, I see a lot of people here, proposing the same thing: -you have it all wrong, you need a rewrite, you need this, you need that. They are all bringing up their 'experience', totally oblivious about the counter-party, totally irrelevant to the question, sounding exactly as you just expressed it - cocky, and nothing useful.
No surprise so the people in this field have such a poor rap.
wow, another genius. So I am waiting for the email.
Money well spent when I spend it on a vacation, everything else is a waste.
As to hiring a consultant - for what purpose? To solve a problem that happens 1-2 times a month on a system that is being used by 50 people 12 hours a day? I already have a solution, and it doesn't involve buying into nonsense.
ha ha ha, well, anything to skip looking at your pathetic faces.
We understand the question, but the answer is your architecture is poor.
- all architecture can be improved.
Killing the user's request will only anger them
- I have explained a bit about it, the user expectations etc.
it is the wrong way
- are you providing money to buy, maintain, support more hardware and software?
your job is to make an architecture that works
- I explained in that link. It works.
and flfills the user's request.
- and I explained in that link, it fulfills them.
My batch-middleware suggestion (which to your clients could be made to look the same as real-time interactive with a job-status page), could even be implemented on the same server
- an irrelevant suggestion. There is no difference between a middleware solution and this one, when in fact all this solution does is generating reports, that's its purpose. All of the resources of the hardware are allocated to it. Every day of the month, 12 hours a day, 50 people use it to generate reports, that's all it's used for. 1-2 times a month (once in January and once in December at this point, from memory), the thing runs out of memory and CPU due to too many abandoned threads. As I explained in that link, near all requests finish under 7 minutes. Very few finish under 10, but that's the maximum, we shouldn't allow requests to run past 15, nobody is waiting past 10 anyway.
There is nothing that middleware would provide, except slowing everybody down in 99% of cases, because it would rely on queues, it would synchronize the work, most of which doesn't need to be synchronized. The problem presented? Stopping some abandoned work threads from going past 15 minutes.
The solution given on this site? Rewrite everything. The reason? apparently 37 years of experience. Too bad none of it is in business.
Beautifully done, I love /. an entire number of assumptions, not backed by any reality, but made with complete certainty. It's an amazing site and sight.
1. I am no longer looking for a Tomcat provided solution, I am quite certain all of the changes must be in application. :) working merrily on an AT&T-Canada project to process large files supplied by an old mainframe that had to be shipped to rebillers, I still remember the first time I had to use java, yes. But thank you.
2. Would you like to pay for the new tool? The transitioning to the new tool? Maintenance? Support? Training?
3. Are you sure that your tool would do a better job than what is currently done with that same data, regardless of how the data is structured? Good to be so sure. I wouldn't be so sure, but that's because I know what the data is, which parts of data are cached, what the structure is. Also, I have this tingling sensation in my left foot, can you tell me if a white pill would do more good for that than this green pill?
4. You are sure that you know everything about the requirements here, so your educated guess about my platform must be right, obviously.
5. I certainly know that I am not going to find any good answers on this forum, but that's not going to stop me from posting. I have an hour to waste.
6. I was there since java 1.0
7. I am sure that you have many many more wonderful suggestions and they are about to come.
The short answer is that a thread can never be both arbitrarily and safely stopped. That's why Thread.stop() has been deprecated since nearly its introduction in Java.
- yes, I have been working with Java since 1997, I have already discovered this, thank you, you know.
Yes, in my case the threads are not blocked, they are doing whatever work they must, running hundreds, to thousands of SQL requests, yes I absolutely can have them implement their own way of timing out because of this specific way they are doing work.
Yes, my question was about whether Tomcat is doing anything about this, providing any interfaces/interrupting listeners for it, or whether this has to be done by the application on its own. So you believe that Tomcat is not doing anything in particular to address this, I couldn't find anything in Tomcat that did it so far, so I am beginning to believe that's how it is.
Thanks again.
Yeah, well, you took the bait and fed the trolls with all the cussing.
- wasting some time, no question about it.
If an in-memory cache is too hard to work out, then just consider a "job" metaphor.
- I have 15G of RAM, 50 users. All product definitions are cashed, but cashing all receipts from all stores and years, etc. and still to allow the container to have enough space to generate all the reports.... I'll need much more space than that, an order of magnitude more space than that.
Whether it's your users resubmitting or a refreshing, doesn't matter: you're making your users wait too long and they think their connection has stalled so that's why they're trying again.
- the users know what amount of time their reports take normally. They know which reports take a few seconds, they know that sometimes they need to wait a few minutes, that's just normal for them, they've been using the system for a year.
In our case 70% of all requests come back in under 10 seconds, 25% last between 10 and 120 seconds, the last 4.99% take up to 7 minutes and then there are odd cases, when users set up a report that normally would return in about half an hour. Our system is used to generate very large reports, in fact before this system the users could wait for their reports for many hours, sometimes days, so they are spoiled now, but not 'Google' spoiled. Saying that "I should never" make a user wait for more than 20 seconds :) Have you ever dealt with retail?
But, if two or three concurrent requests can clog/kill your server, you're doing something wrong.
- as I said, we have 50 people working in various geographical locations and this problem comes up 1-2 times a month, a very specific set of circumstances.
Also, training your users that your webapp just "takes a long time" isn't a very effective approach - grumbly users will turn on you eventually. It's your job to keep users happy and keep your reputation spot-free.
- thank you for telling me my job, but you don't know my job :) I love /.
I agree with the other poster, perhaps you need to invest in a BI solution. Or, take the graph generation process offline with queues and email if BI's out of your range. What you are describing does not sound workable; trust me, it's rethink time, and there's no shame in that because you're making things better.
- I don't know why you are talking about shame, I have no reason to get anything new to solve this problem. It would be very silly and ineffective to buy more hardware and to transition all of the reports from one system to another, when in fact the entire system is created just to generate the reports.
The only issue that exists in this system, which works fine for the 50 users we have every day of the month, 12 hours a day, is that 1 or 2 times a month too many reports are created at once that are dumped and re-created again by enough people that the VM itself starts running low on memory and CPU, and this can be easily fixed by killing a few threads that go overtime.
Imagine you own a piece of equipment that does everything you need, much better than a piece of equipment you used to have for years, it's orders of magnitude faster, it's actually distributed geographically while the old systems weren't, it is really fast compared to previous systems, yet once in a while the system goes off-line simply due to too many requests that are not done yet, but that are really just run abandoned. Imagine that you are not 'Google', you are not going to be 'Google' either. It's a normal situation.
To come to a business and say: because of this specific issue the entire system needs to be recreated in some other way, rewritten, new hardware needs to be bought, supported, etc. That's all great, but the question is simple: why? It's a silly suggestion, no offense, but it's obviously made by somebody not running a business.
You can go ahead, asswipe, use your tongue.
Yeah, I don't care about /. AC crowd, they are clowns.
As to the issue at hand, it's not about the 'refresh' button, it is just people abandoning one report to try and generate another and it's not a normal occurrence, not something that happens all the time, but 1-2 times a month, hard to spot and slows everything to a halt. Normally it happens when the users start generating yearly graphs with too many data types on them, like sales for all products within a number of brands on a comparative graph over a period of 2-3 years for around ten stores at the same time.
So even a graph like that only takes maybe 10 minutes to create, but if they decide they want to change the parameters of the graph and abandon the first one to start generating a second, a third, and we have up to 50 users at the same time, all of a sudden there is a spike that's not normal for the app, and the tasks are still all going at the same time and the system runs out of memory and starts thrashing threads it looks like.
Unfortunately cashing responses is not what I can, since every report can be generated with tens of parameters, for arbitrary sets of stores, for arbitrary periods of time, etc. Also they are aggregate in many cases, what I means is that sales are added together in the DB based on filters, so again, I can't pre-generate discrete pieces of it to just join them together for a request.
So since this happens only on rare occasions I was thinking about timing out threads that are taking too long, that's an easy cop out, maybe just time out any task that goes on for more than 15 minutes, something like that.
I used to do that earlier - the main request thread creates a worker thread and is put to sleep for a short period of time, it wakes up, checks if the worker is done, if it's not and if it's not the timeout yet, it goes to sleep again, if it's past time out, it kills the worker thread and returns with application timeout error. It works, but too bad Tomcat doesn't allow for that itself.
Another moron with a keyboard. Polling connection on what, you, nimrod? From the client it's useless, the thread is already running in the container, nothing can stop it until it's done with its task. From the thread itself - the client can still be there just fine, working on some other report. Go bang your stupid fucking head against the wall, maybe you'll set your brains straight.
First: it's a normalized database.
Second: nobody can answer a simple question, is it possible to time out a running task through Executor thread. Your solution is not doing much good, is it? Since the user can have multiple reports running at once legitimately.
Third: I don't need your implementation, since you don't understand the question.
excellent, all of ACs can go fuck themselves.
Didn't I tell you to go fuck yourself in enough ways? Can't you take a hint?
If you are the original asshole (all of you, ACs, look the same to me,) then go fuck yourself. You are the one who was being the ass.
Not good. The report interface is dynamic, allows the users to change filters and ordering in real time, the entire point of this app is to do this in real time. The only problem is they are not limited on what they can do in terms of the total size of the data set. Normally even the largest requests return within 2 minutes, it's actually good enough for them.
However in cases of graphs, they can choose too many data types to be on them. For example they can choose not just brands to be on the graph, but different products under brands to be all displayed all at once for a very large time period. It's not an unusual thing and they want the result in front of them once it's compiled. Even these things return quickly enough (under 7 minutes normally). But the problem appears sometimes when they start a graph, don't wait for it to finish, change some parameters and restart the graph, etc.
The server has 16G of ram, 15G are allocated to Tomcat, so there is no memory left to do these graphs somewhere else, they actually take a lot of space as well.
I am constrained on the total memory, on CPU, on total time, basically on everything, those are hard constraints, it's not like I have a bunch of servers standing there, only one. So no middleware, nowhere even to run it, and the graph is not good if it comes too late.
Again, the question is much simpler: is there a way to time out Executor threads in Tomcat or not?
If not, then that's another reality and I have to work around it. It is possible to work around it - it's possible to stop execution of a task in the middle between SQL requests. It's possible to have do what I did sometime earlier - start an asynchronous thread for each web request, put the current thread to sleep for a small amount of time, once the time passes and first thread wakes up, check if the worker thread is done.
This way I have the time out that I can enforce - I can kill the worker thread I started.
My question is about Tomcat Executor threads - can they be killed from the app? Can they be configured to time out?
I think my question is very simple, it doesn't require anything beyond those very simple specific answers, doesn't look like people understand the questions, everybody is trying to re-architecture everything or to be a smart ass (as the rest of the thread clearly shows.)
ha ha ha ha, what a dumb ass, still not getting the question yet offering a useless, clueless comment here.
Yet with your 12 years you still didn't get the question, did you? How do you time out an executor thread that's no longer useful, if it's still doing work, type of work that takes long time to do? - so if this looks like an idiotic question to you, then you haven't had to do much in those 12 years.
Secondly, it's not 'refreshes' that cause this, it's dumped requests, that are still processing. Sometimes many dumped requests.
Clearly there can be more done, more memory, a cluster, etc. Most of it would be a waste because this particular issue is only with threads that are still going, sometimes going for 45 minutes, taking up processor and memory.
So in reality, if this is not though through by the Apache team, then the question is: have they ever had to write code to produce very large reports on very large data sets and if not, why not? I am sure this is not the first time somebody is doing it in real time.
The 45 minute requests are a rarity, all of the parameters are user set, so if a user chooses a very large data set to go through and then decides not to wait for it, or tries to change the parameters of the request and restart it again - that's a normal use case.
Yes, some reports take thousands of SQL executions and massive amounts of data to be held in memory. No, this is not an every day occurrence. Yes, there are more than 1 user, around 50 people doing various things. Yes, it would be nice to have a cluster, but most of the time this is not an issue so it's not really a hardware problem.
It is possible to have checks in the code itself, and verify between executions of SQL statements whether this request should be terminated, so that's one way of doing it. But if the servlet container is not helping by providing at the very minimum a time out mechanism for the Executor threads then why not? It should be.
--
So in short, with your 12 years of experience, you are the one who looks like a dumb ass to me if you can't imagine a situation where Executor thread timeouts could be used for good before ideas like clustering and distributed nodes are considered.
ha ha ha ha, idiot, didn't I say: what is the way to time out an executor thread? What is the difficult thing to understand? You can sit there and yap and yelp about code as much as you want, but what is the way to kill a thread that is taking too long? How the fuck, do you, idiot, know what a thread is doing? It's creating a gigantic 3 year based report by running SQLs against a dabase and putting together a picture that's going to be placed an HTML page.
Clearly the web is screwed up so there is no state in HTTP and the server can't easily know that the client is no longer waiting for the response, but there has to be a way to deal with requests that do not need to be executed any longer.
So, idiot, again, yap away, or provide a solution based on Tomcat configuration, otherwise there will have to be plenty of code written.
Come on, do something USEFUL once.
yeah, yeah, asshole. You have any idea how to time out an executor thread after an amount of time has passed, or no? If not, shut your yap.