I too, was scared of the unknown some 35 years ago when we first were going to become LP users. After having been a steady user since. I now wonder why I was concerned.
BUY a tank. If you own a tank you are not stuck HAVING to buy from one place, so the prices Fred talked about will vary up to 3 bux a gallon, yes at the exact same time. Supposedly there is regulation, I cannot prove it, it seems to be a whatever the market will bear, but then some guy has top grade fuel and is selling for peanuts, when most of the competition is selling like it's in short supply. I would also suggest a 1000 gallon tank, It doesn't go bad.
Treat your tank well. If you mount above ground do it on a minimum 6" thick concrete pad, with at least a foot of crusher fill underneath. It doesn't so well to buy, and have the weight of a full tank cause you to go off the pad, and lose $$$$$. If you bury, check around and get the best excavator who has a lot of experience putting in tanks, DO NOT let a gas company do it, they are hacks that just sell gas, and your tank is liable to float out of the ground with a heavy rain.
ALWAYS steer your purchase toward July, gas is historically much lower then, you can map this if the locals allow you to see pricing. This year I bought at 58 cents a gallon, tax, and delivery included. I had to buy in January once, and Yikes!!!!!! OUCH.
I was initially worried about some ticking time bomb near my house. Seriously, when is the last time you heard of a LP tank going boom. In our first home we had natural gas, and we used to smell gas stink all the time. With LP I have never smelled gas. It likes it inside the tank, and pipe.
I wouldn't be comfortable though with Pex, it has too much crush possibility. Possibly those saying PEX are thinking the Stainless Steel jacketed line used for direct burial is PEX because it has a plasticy yellow jacket. Ultimately you should only have about 15 to 30 feet of line from the tank, to the heat source. If your home is on something else for heat, just put the tank at the shop. I have a 1000 gallon tank buried at the house, and a 500 on a pad at the shop, seems to work pretty well.