Hi Folks
I'm working on an old legacy app and noticed something. It caches a bunch of info (lookup table data) from the database using a ServletContextListener. I think opening DB connections in a listener is reasonable. While there is no business logic in the listener, I'm not sure doing a bunch of DB heavy-lifting operations in a context listener is a "good thing", although I don't really have a concrete reason why. Perhaps I'm just being fussy. Anyway, in your opinion: 1. Is performing DB heavy-lifting operations in ServletContextListener a "reasonable" practice? 2. Is there a "better" way of caching said items at application startup? Thanks for your time and consideration. :-) -- Cris Berneburg CACI Senior Software Engineer ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. |
Cris,
On 4/8/21 07:54, Berneburg, Cris J. - US wrote: > Hi Folks > > I'm working on an old legacy app and noticed something. It caches a bunch of info (lookup table data) from the database using a ServletContextListener. I think opening DB connections in a listener is reasonable. While there is no business logic in the listener, I'm not sure doing a bunch of DB heavy-lifting operations in a context listener is a "good thing", although I don't really have a concrete reason why. Perhaps I'm just being fussy. > > Anyway, in your opinion: > > 1. Is performing DB heavy-lifting operations in ServletContextListener a "reasonable" practice? > 2. Is there a "better" way of caching said items at application startup? > > Thanks for your time and consideration. :-) IMHO there is no better way than using a ServletContextListener to load things at startup. Your only other spec-compliant option is to use a Servlet with load-on-startup set and do your work in the init() method, which is ... ugly. SCL is the way to go, here. Another option would be to perform "lazy loading" instead of a-priori loading of this data. You will take the hit of loading the data when it is first requested, which may negatively impact user experience. It might also mean that you have to be more careful about cross-thread synchronization, etc. since you can't guarantee that the work has already been done before a client tried to access the cache. If you are concerned about startup times, lazy-loading is a good solution. It can also improve your memory usage if that data is never actually needed. We have a primary application at $work where we need to have a lot of information in mrmoey to be able to do important stuff. This information evolves over time and has a long (essentially infinite) history. We can never really get rid of anything entirely, but the older some of that data gets, the less likely it is to be used. We loaded 100% of it every time at startup. At some point, it started taking several seconds to launch the application, then eventually several tens of seconds and I decided it wasn't tolerable anymore. So I switched to loading things on-demand and it made not only a significant performance improvement on startup (hey! we are loading nothing on startup now! super fast!), it significantly reduced the memory footprint of the in-memory cache of data because that rarely-used older stuff was just never being loaded at all. To be sure, it was a significant change to our application, but one well worth the investment. (We also have a user-initiatable process to "reload" the data that was taking the same amount of time. Not it just empties the cache and does nothing else. More faster. :) -chris --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Berneburg, Cris J. - US-2
Hi,
What happens when the DB has problems when the webapp starts? Will the startup fail then? I think doing lazy init is the better approach, when db comes back it will work again after the webapp did start. Mfg Thomas Am 8. April 2021 13:54:46 MESZ schrieb "Berneburg, Cris J. - US" <[hidden email]>: >Hi Folks > >I'm working on an old legacy app and noticed something. It caches a >bunch of info (lookup table data) from the database using a >ServletContextListener. I think opening DB connections in a listener >is reasonable. While there is no business logic in the listener, I'm >not sure doing a bunch of DB heavy-lifting operations in a context >listener is a "good thing", although I don't really have a concrete >reason why. Perhaps I'm just being fussy. > >Anyway, in your opinion: > >1. Is performing DB heavy-lifting operations in ServletContextListener >a "reasonable" practice? >2. Is there a "better" way of caching said items at application >startup? > >Thanks for your time and consideration. :-) > >-- >Cris Berneburg >CACI Senior Software Engineer > > >________________________________ > >This electronic message contains information from CACI International >Inc or subsidiary companies, which may be company sensitive, >proprietary, privileged or otherwise protected from disclosure. The >information is intended to be used solely by the recipient(s) named >above. If you are not an intended recipient, be aware that any review, >disclosure, copying, distribution or use of this transmission or its >contents is prohibited. If you have received this transmission in >error, please notify the sender immediately. -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet. |
In reply to this post by Christopher Schultz-2
Hey Chris
cb> 1. Is performing DB heavy-lifting operations in ServletContextListener a "reasonable" practice? cb> 2. Is there a "better" way of caching said items at application startup? cs> IMHO there is no better way than using a ServletContextListener to load things at startup. OK, good to know that using SCL is "reasonable". cs> Your only other spec-compliant option is to use a Servlet with load-on-startup set cs> and do your work in the init() method, which is ... ugly. I was thinking of a servlet request (or something) that is called on startup that could also be called later on-demand(?). cs> Another option would be to perform "lazy loading" instead of a-priori loading of this data. cs> You will take the hit of loading the data when it is first requested, which may negatively cs> impact user experience. It might also mean that you have to be more careful about cs> cross-thread synchronization, etc. since you can't guarantee that the work has already cs> been done before a client tried to access the cache. cs> If you are concerned about startup times, lazy-loading is a good solution. cs> It can also improve your memory usage if that data is never actually needed. +1. I like this. "Smarter" caching. Only load the data you need when you need it. cs> We have a primary application at $work where we need to have a lot of information cs> in mrmoey to be able to do important stuff. [...] We loaded 100% of it every time at startup. cs> [...] I switched to loading things on-demand and it made not only a significant performance cs> improvement on startup [...] it significantly reduced the memory footprint of the cs> in-memory cache of data How were you "careful about cross-thread synchronization", synchronized blocks? cs> We also have a user-initiatable process to "reload" the data Where do you do the loading and reloading, a in a servlet request? cs> [Now] it just empties the cache and does nothing else. More faster. :) "More faster" :-) -- Cris Berneburg CACI Senior Software Engineer ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Thomas Meyer
Hi Thomas
Thanks for the info and your opinion! :-) cb> 1. Is performing DB heavy-lifting operations in ServletContextListener cb> a "reasonable" practice? cb> 2. Is there a "better" way of caching said items at application cb> startup? tm> What happens when the DB has problems when the webapp starts? tm> Will the startup fail then? Good question. I don't know, but I would guess it would fail. Or the web app would be in an unusable state since the needed cache would be empty. I think at that point the app would need to be restarted. tm> I think doing lazy init is the better approach I'm starting to agree. :-) tm> when db comes back it will work again after the webapp did start. So the web app would be more "robust" - it would cache the data when the DB is back online. Sounds good. :-) Hmm... I'm kind of undecided about this. If the DB is down during startup, then the web app would be unusable anyway. Would it not make sense for the app to be down too? I guess it depends on how the app handles DB connections and errors? If the app: * Displays a generic "System is down for maintenance" message when the DB is inaccessible and prevents the user from clicking things. * Versus displaying weird messages to the user whenever they click a button or link but does not stop them from trying. -- Cris Berneburg CACI Senior Software Engineer ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Berneburg, Cris J. - US-2
Cris,
On 4/14/21 08:03, Berneburg, Cris J. - US wrote: > cs> Your only other spec-compliant option is to use a Servlet with load-on-startup set > cs> and do your work in the init() method, which is ... ugly. > > I was thinking of a servlet request (or something) that is called on > startup that could also be called later on-demand(?). How would you trigger that servlet to be called on startup? Some kind of script that does catalina.sh && sleep $time && curl http://example.com/load-stuff ?? How would you determine the value of $time? What if it fails? > cs> We have a primary application at $work where we need to have a lot of information > cs> in memory to be able to do important stuff. [...] We loaded 100% of it every time at startup. > cs> [...] I switched to loading things on-demand and it made not only a significant performance > cs> improvement on startup [...] it significantly reduced the memory footprint of the > cs> in-memory cache of data > > How were you "careful about cross-thread synchronization", synchronized blocks? That's one way to do it. You can also use thread-safe classes which either implement their thread-safety in one of a few different ways, synchronized blocks being one of those strategies. > cs> We also have a user-initiatable process to "reload" the data > > Where do you do the loading and reloading, a in a servlet request? Yes, but the idea there was to "freshen" the data from the database if it had been altered by some other process e.g. an update from a database where new content is added, then migrated into production via direct SQL drop. So it really was a "reload" operation. These days, it's an "unload" operation. :) -chris --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Hi Chris
cb> I was thinking of a servlet request (or something) that is called on cb> startup that could also be called later on-demand(?). cs> How would you trigger that servlet to be called on startup? cs> Some kind of script that does catalina.sh && sleep $time cs> && curl http://example.com/load-stuff ?? cs> How would you determine the value of $time? What if it fails? Pfft, beats me. :-) I was just grasping at straws, apparently. cs> You can also use thread-safe classes which either implement cs> their thread-safety in one of a few different ways, synchronized cs> blocks being one of those strategies. Got any buzzwords for me that I can lookup "one of a few different ways", other than synchronized blocks? cs> "freshen" the data from the database if it had been altered by cs> some other process e.g. an update from a database where new cs> content is added, then migrated into production via direct SQL cs> drop. So it really was a "reload" operation. These days, it's an cs> "unload" operation. :) I was wondering about that. Sounds like it basically invalidates the cache so it can be reloaded later when needed. -- Cris Berneburg CACI Senior Software Engineer ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Free forum by Nabble | Edit this page |