I am no expert on the toxicity of anything including gallium arsenide, but it is important to distinguish the safety issues with both the manufacturing process and in a final product. I suspect there are issues with the manufacturing process with gallium arsenide, just as there are issues with the manufacturing of semiconductors with silicon (for example, the chemicals used in the process - google it, there are a lot of issues with silicon). I would think there are easy ways to deal with the safety issues with the manufacturing process, with simple personal protective equipment and/or environmental controls. Also google lawsuits/silicon valley/usepa/contaminated land/etc.
For example:
http://www.sfgate.com/bayarea/article/The-valley-s-toxic-history-IBM-trial-is-latest-2826844.php
The other issue to consider would be the toxicity in the final product. The gallium arsenide would need to get in you somehow to be toxic. It is unlikely that there would be any inhalation issues as there are no volatility issues at all (the boiling point of gallium arsenide is 1238 C). So then I would say that you would need to ingest it for it to be a problem. Given this, perhaps some simple instructions would be "don't eat the chips". I would also recommend not eating any batteries that contain cadmium, or any light bulbs that contain mercury. Seems do-able as there are plenty of other tasty things that can be eaten.
Good luck!!
Quan