Absolutely overclock! Intel doesn't have time to figure out what the max achievable frequency is on every chip they make so they guarantee a minimum and let you figure out the max. There are a billion (or so...) YT videos on how to do it but basically you keep adjusting the "multiplier" and Vcore (core voltage) in the BIOS while keeping an eye on tempartures through something like HWMonitor or some other free app. What is a good temp and/or voltage will depend on the processor but there are another billion or so references that can give you guidance specific to your chip.
It takes an hour or so but can greatly increase your processing power and real-time performance, so definitely do it.
Detractors say that it decreases the life of a chip, and that's true, but unless you're running 80%+ CPU usage 24 hours a day then you'll never notice. I've been overclocking for decades and have never had a chip die. If you're a composer/producer running a few machines they get replaced long before they die. If you're running 500 machines in some kind of server setup then yeah, longevity is an issue.
And unless you're doing something crazy like liquid nitrogen cooling then you really can't damage the chip. It'll protect itself by crashing. Then you reboot, drop the multiplier, and try again.
Just to add to @rgames' great points - Definitely overclock. Not every chip is exactly the same. So you might find a video or a resource that shows your chip overclocked to X.X Ghz, but yours can't - or maybe it does even better.
More Vcore voltage = more heat. Strive to find a good balance of clock speed and heat.
For example, my 7700k is stable at 4.8ghz at nearly stock vCore voltage. But in order to get it to 5.0ghz I have to increase the vCore significantly. To me, the extra heat is not worth the 200mhz in performance.
Lots of people still using 10 year old processors overclocked like they are brand new.