The ASRock Z77 Pro3 has three PCI Express slots. The first is an PCI-E x1, this is intended for things like add on sound cards and the like, you will probably not use this slot. The second slot is a PCI-E 3.0 x16 slot, this slot is intended for your graphics card. PCI-E 3.0 is backward compatable with PCI-E 2.0, so your graphics card will fit and function in this slot. You will want to ensure that you leave the BIOS setting that contolls the configuration of this slot set at PCI-E 2.0. The third PCI-E slot is version 2.0, it is physically large enough for an x16 card, but only has 4 lanes of electrical signaling enabled, hence the x4 label. This allows you to install a card such as your graphics card in this slot, and it will work there, but with a reduced data rate and thus lower performance.
Bottom line, your card should work just fine in this motherboard, install your graphics card in the "first" (nearer to the CPU) big PCI-E slot.