I learned from this site that the bullet pencil "came about during the civil war. Soldiers would stick a pencil
in the end of used rifle cartriges so the tips would not break off while the pencil was stuck in their packs."
 

(click for full-size image)

The oldest pencil has a slightly narrower neck, and a slightly longer top (23 mm versus 19 mm)

The oldest pencil also does not have the smaller center circle on the bottom, indicating where the primer would be.

A few have erasers, but note that these are the ones with a steel rather than brass neck.

(click for full-size image)

I believe the original pencils were round, the replacement pencils have the more modern six-sided cross-section.

(my newest acquisition, with a unique red-painted neck, the body including under the paint is brass not steel)
 
Here is an interesting advertisement from the 1948 Reunion Booklet, on page 42 listing the company manufacturing/selling the Pencils to the 314th:
 

 
Even earlier on the 1936 financial statement we see mention of 1,534 penciles in inventory - purchased at a cost of 10 cents each!

 
Perhaps the world's largest collection of 314th Infantry Memorial Log Cabin Bullet Pencils? (more than 100)
 

 
Another 314th Infantry - Log Cabin Memorial - souvenier was a Celluoid Mirror
(front shown above, reverse is the mirror)