At first it takes the stored tiles, so if you have stored 10 tiles and 5 breakes attack you they cannot harm your island!
Next it takes the tiles from the attacked side. Example: If the attacker is standing left from you, the breaker takes the tiles from left side of your island.
The best way to be safe is to fortify your island. You don't have to fortify every tile, you can fortify the cost and if you have closed the ring, the inner tiles are automatically fortified.
For the times, you aren't (complete) fortified it's helpful, to watch your neighborhood. Islandbreakers are very slowly and the higher the range between attacker and you is, the more hours!!! they need to arrive. (for the 7:30 attack the range must be under - I think -