Building a product as a side project, while keeping up with your day job and caring for your family, is the ultimate time-management challenge. That much I have already learned since starting work on Coursify. I have also learned that I am not alone in this challenge. For instance, check out this post by Craig Mankelow.
Essentially, as with everything in life, you have to pick priorities. How important is this to you? What are you willing to sacrifice? Start with easy, obvious stuff: TV, video games, sleep (only occasionally). It's tougher than it sounds, as you need to make a habit out of this and old habits will put up a fight before being definitively abandoned.
There is also another caveat to this tip. Entrepreneurs need to be creative, and only people who actually live fulfiling lives can be creative. Creativity stems from joining together different bits of information and experiences, therefore if you do not provide yourself these experiences you will fail to be creative. This means that while you do have to carefully manage your time, you cannot simple abandon every single activity not directly related to your dream.
Overall, remember the famous sentence not said by Albert Einstein:
Insanity is doing something over and over again and expecting the same result.
If you want extraordinary results from your product/business, do extraordinary things to get there. You might even fail anyway, but you'll have fun in the process.