Whenever an ore vein refreshes after being harvested from, there is a slight chance it will change resource type. For example, a mountainside that yielded Valorite today might give plain iron tomorrow, and vice versa. Your level of skill does not determine your chance of finding a colored vein, it only determines whether you can extract the colored ore.
So if you completely empty a vein everytime it has a slight chance to change the ore type in that vein.