Current-induced spin-orbit torque (SOT) switching in a heterostructure with perpendicular magnetic anisotropy (PMA) has attracted great attention as a new writing method for spintronic devices. However, this highly attractive switching scheme is often accompanied by an unfavorable external magnetic field. In this work, we demonstrate polarity-controlled field-free SOT switching in 3d Cr. Moreover, we find that Cr metal can induce strong interfacial PMA, without either heavy metal or MgO layer. Most importantly, field-free SOT switching could be achieved without introducing asymmetrical geometrical pattern, heavy metal, additional ferromagnetic or antiferromagnetic layers. We show that the underlying cause for the deterministic field-free switching lies in the slanted columnar microstructure, whose tilting angle is only around 5°, for the otherwise uniform thin films. The direction of oblique columnar structure dictates the up and down orientations of the PMA layer. Our results uncover the significant role of 3d materials and shed light on field-free SOT magnetization switching.