Please don't anyone think this post contains any amount of fact.
Sorry, you're not born with any penis cells. You basically start as a blob of undifferentiated cells, as your shit progresses, your cells begin to differentiate into specific functional roles.
As far as genes go, its not black and white. Some things, such as hair on your knuckles, attachment of your ear lobe (some people have a hangy flap, some don't) are directly related to your father's genes. If he had it, you have it, vice versa. The vast majority of human characteristics are not that simple. Its one giant gray area. You can be predisposed genetically, but you may or may not display specific traits. These discrepencies can be attributed to environmental factors, or just plain random chance (random as far as we know at this point).
What genes are passed on generation to generation also has alot to do with where the gene is physically located on each chromosome. Recombination, crossing over, etc. all plays a part. If your father has a gene on his X chromosome, you're not going to inherit it (talking about guys here). You get your X from your mom, Y from your dad.
Its not nearly as simple as saying some traits are dominant, others recessive, and if we know your family history we can use punnet squares to figure out the chances that your kids will have specific traits. Would be interesting if that were the case though
If you're really interested, pick up a genetics book and try reading it. I dunno of anything thats written simply enough for most people to understand without taking classes on the subject though. Worth a shot if you really want to try and understand the way shit works.
P.S. i've been giving alot of medical advice around here lately. For those curious i'm a medical intern in nuclear medicine. I'm not a reproductive specialist, i'm just trying to share some of my knowlege i've picked up over years of pre-med and med courses. I'm no expert.