Từ giả thiết ta có
+ $x^2+y^2+xy = (x+y)^2 -xy = 1 \Rightarrow (x+y)^2=xy+1 \ge 0 \Rightarrow xy \ge -1$
+ $x^2+y^2+xy = (x-y)^2 +3xy = 1 \Rightarrow (x-y)^2=-3xy+1 \ge 0 \Rightarrow xy \le \dfrac{1}{3}$
Vậy $-1\le xy \le \dfrac{1}{3} \ (1)$
$A=x^2+y^2-3xy =(x^2 +y^2+xy) -4xy=1-4xy \ (2)$
Từ $(1);\ (2) \Rightarrow -\dfrac{1}{3} \le A \le 5$
+ $max A = 5 \Leftrightarrow xy=-1 \Rightarrow x^2 +y^2=2 \Rightarrow (x;\ y)=(-1;\ 1);\ (1;\ -1)$
+ $min A =-\dfrac{1}{3} \Leftrightarrow xy=\dfrac{1}{3} \Rightarrow x^2 +y^2 = \dfrac{2}{3} \Rightarrow (x;\ y)=(-\dfrac{1}{\sqrt 3};\ -\dfrac{1}{\sqrt 3});\ (\dfrac{1}{\sqrt 3};\ \dfrac{1}{\sqrt 3})$