This advice is specifically for Comcast users but some of the points are useful for other ISPs. Personally, I look at the cable modem to determine if it is reducing my network flow. Though Comcast tends to use old equipment. The Wi-Fi usually combined in a router can slow you down. It’s not just Comcast’s equipment Wi-Fi but it’s your neighbor’s Wi-Fi that can be problematic. Consider how much internet you actually use. Count every device that’s connected to it and if you can access the management panel you can often see the full list of devices.
The bigger issue is around security and administration. I think the administration of a Comcast Xfinity device is a little bit easier because they give you an app but it’s still not simple and I don’t really trust their firmware.