Private Domain Registration Vs Public…is Private Registration Important?

I’m purchasing a domain name from godaddy and I’m confused as to whether or not paying extra for a private registration is worth it. I understand that if I list under public it will be published in the whois directory but I’m not sure I see the advantage of listing private. Is my business address and name the only thing I’m hiding by going private or is there more to it than that? I would appreciate anyone who could clarify this. Please keep in mind that I’m not the most savy with technology “lingo”. Thanks!