I actually thought about it for a very long time. The problem was when I was researching a way to do it, I wasn't convinced that I could build something that would hold up to rocks and bricks slamming into it. The cart is light enough that even full of small boulders I can still flip it over and wheel it about by myself without to much trouble.
That garden cart is actually very decently priced. Upon unboxing, I noticed it wasn"t really that heavy as I thought. Its actually pretty light and easy to carry and it has a plastic base. This is the lower end version of the Gorilla cart as there is also a higher end version that comes with a much more sturdy base. https://www.ronmowers.com/lawn-mower-attachments/dump-carts/