Bring chicken broth to a boil in a large soup pot then add cauliflower. Place lid on top then cook until cauliflower is fork-tender (don't overcook.) Meanwhile, melt butter (or extra virgin olive oil, if using) in a small skillet over medium heat then add garlic and saute until golden brown, about 1 minute. Carefully transfer cooked cauliflower and 4 cups chicken broth, reserving remaining cup broth, to a blender (may need to do two batches if blender is small) then add garlic butter, milk, and salt. Blend until very smooth, 1-2 minutes, then set aside. In the same soup pot, cook bacon over medium heat until crisp. Transfer to a paper towel-lined plate to drain then remove all but 2 teaspoons bacon grease. Add carrots, celery, and shallots, season with salt and pepper, and then saute until tender, 5-6 minutes. Add cauliflower sauce, reserved cup of broth, and hash browns to the pot then place a lid on top, turn heat up to medium-high, bring chowder to a bubble and cook until potatoes are tender, 8-10 minutes. Add sweet corn then cook until warmed through. Stir cooked bacon into chowder then ladle into bowls.