ORA-15067: command or option incompatible with diskgroup redundancy

At our company we had to replace the storage cabinets of a two node RAC cluster. The RAC cluster uses ASM diskgroups with NORMAL redundancy.
All diskgroups exist of two failgroups, with each failgroup located on one of the storage cabinets, for redundancy purposes, as seen here:

SQL> select   distinct a.name , b.failgroup, count(*) numdisks
  2  from     v$asm_diskgroup a
  3         , v$asm_disk b
  4  where  a.group_number = b.group_number
  5  order by name;

NAME         FAILGROUP             NUMDISKS
------------ --------------------- ----------
DB01         DB01_0000                     14
DB01         DB01_1000                     14
FB01         FB01_0000                      5
FB01         FB01_1000                      5

While all disks in FAILGROUPS DB01_0000 and FB01_0000 are on one storage cabinet the command to drop all disks on one 'side' of the ASM diskgroup at once would be the first option to choose:

However:

SQL> alter diskgroup fb01 drop disks in failgroup FB01_0000;

alter diskgroup fb01 drop disks in failgroup FB01_0000

*

ERROR at line 1:

ORA-15067: command or option incompatible with diskgroup redundancy

Since all data is mirrored across the two failure groups, one would expect to be able to drop one complete set of the mirror, so one complete failgroup.

All can be explained by the redundancy of the diskgroup. Using NORMAL redundancy, the diskgroup expects to have at least one disk available in the first failgroup that can be used as a destination for a failing disk in the other failgroup. This is also seen when querying the REQUIRED_MIRROR_FREE value of the diskgroup:

SQL> SELECT name, type, total_mb, free_mb, required_mirror_free_mb,usable_file_mb FROM V$ASM_DISKGROUP;

NAME                           TYPE     TOTAL_MB    FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB

------------------------------ ------ ---------- ---------- ----------------------- --------------

FB01                           NORMAL     512000     503598                   51200         226199

DB01                           NORMAL     819200     801612                  102400         349606

The REQUIRED_MIRROR_FREE values are equal to the size of a single disk in these diskgroups as seen here:

SQL> select distinct * from (

  2    select b.name, a.total_mb

  3    from v$asm_disk a

  4  , v$asm_diskgroup b

  5  where

  6  a.group_number=b.group_number);

NAME                             TOTAL_MB

------------------------------ ----------

DB01                               102400

FB01                                51200

So the NORMAL REDUNDANCY prevents us from dropping all disks at once in a failgroup.

Solution:

Drop all disks but one in the failgroup and then first add at least one new disk from the new storage cabinet

After that dropping the last disk from the old storage cabinet should be possible.