In my years of homeschooling, I've come across some inexpensive homeschool books that still teach what your child needs to know without a lot of the fluff or marketing that other materials have.
Besides purchasing books used or using the library and the Internet, there are some book publishers that have found ways to produce inexpensive materials that homeschoolers can use.
Spectrum produces some nice workbooks that are about $7-9 that you can use for an entire year per subject. They carry Math, Vocabulary, Test Prep, Geography, Writing, Spelling, Language Arts, Reading, and Phonics for grades PreK-8th grade. Several of my children have used these books at various times and we have always been happy with them. If you are very limited with funds, you could get these and feel like you have a core to work with. Even if you don't have limited funds, these books are great to use!
Christian Liberty also has a nice line of inexpensive homeschool books. They carry History, Spelling, and Handwriting available at Christian Book Distributors.
Remember that drawing is just as much handwriting practice as writing letters. Their little brains are still trying to get their fingers to do what they want them to do whether they are drawing a picture or writing a sentence. Make sure you have lots of paper, pencils, markers, and any other items to encourage drawing and writing. No need for expensive curriculum here when you have the right supplies that can be purchased cheaply or found for free.
For grammar, we use Mad Libs. They are great for teaching the parts of speech and for lots of fun review. Check online for free grammar and language arts lessons.
For teaching reading I used Teach Your Child to Read in 100 Easy Lessons. I didn't use this necessarily because I was trying to save money, just because it worked. I taught all 4 of my children how to read with this program. I didn't use the writing part though. We just used the pictures and instruction. I don't think any of them ever finished the book. By the time they got to lesson 75 or so, they were moving on to reading other things.
When thinking about inexpensive homeschool books, consider that just because something has a lot or a little marketing behind it, doesn't mean it works any worse or better than any other book or program. Just because you spent a lot of money on a curriculum, doesn't mean your children will have a better education or retain the material better than someone who just had some library books, paper and pencils available.