A RAID controller is not something that everyone needs, but if you are involved in building and maintaining computer information systems, you probably know something about this “wonder of technology”. A few years ago many people had not heard anything about them at all, but today they are present in almost any mid-level server.
A RAID controller is an element of a computer system that provides fault tolerance in case of a disk drive failure and also increases the performance of the disk subsystem. Everyone probably understands why this is important, but let’s look at some aspects of the problem.
In spite of the fact that MTBF of modern high-end discs is enormous (more than 100 years), practice shows that they still fail. There are a number of objective reasons for this – the life of disks is affected by unstable power supply, vibration and on/off cycles, as well as temperature violations. In addition, there is some possibility of factory defects. So, if you want to protect your data and avoid downtime, you can’t do without a RAID system.
As for the performance of the disk subsystem, the problem is that it lags far behind other elements of the computer. The situation can be described as a crisis of I/O of the secondary storage system. Inability to significantly increase technological parameters of magnetic disks entails the need to find other ways, one of which is parallel processing. In some cases, the use of a RAID controller to provide increased performance is even the first priority (e.g., for video editing), and system fault tolerance is a secondary factor.
There are three basic implementations of RAID systems:
- Software-based;
- Hardware – bus-based;
- Hardware – autonomous subsystem (subsystem-based).
Each of the above implementations is based on software code execution. They differ in fact in where this code is executed: in the computer’s CPU (software implementation) or in a specialized processor on a RAID controller (hardware implementation).
Simple RAID levels 0 and 1 are usually implemented in software, as they do not require significant computation, but sometimes RAID 5 is also implemented. Given these peculiarities, software RAID systems are used in entry-level servers. But there are also more interesting software implementations, such as Adaptive RAID, which dynamically changes the way data is displayed depending on its nature and usage dynamics.
Hardware RAID controllers usually implement the full range of standard levels, and sometimes have a number of additional features. The most powerful, high-end systems can auto-configure and automatically select RAID levels and distribute data in real time.
Let’s take a look at the major RAID controller vendors that are available in the distribution and OEM markets.
The powers that be
Among the strongest worlds of PCI-to-SCSI RAID controllers are Adaptec, AMI (American Megatrends), DPT (Distributed Processing Technology), Mylex. Then there is perhaps IFT (Infortrend Technology), better known in the OEM market than in the distribution market. It should also be noted Compaq which, today, is the No. 1 RAID controller manufacturer due to the use of its controllers in its own servers, but only because of this. Vortex can be attributed to sufficiently powerful manufacturers, it has a significant share of the German market of RAID controllers, but outside of it the share of this manufacturer is negligible.
On the world market of distribution and OEM, the most widespread Mylex and AMI, where the first dominates with a significant gap, thanks to well-developed distribution channels.
A slightly different situation with SCSI-to-SCSI controllers. As they are oriented on more expensive and less widespread external solutions, you can see them in the price lists of accessories quite rarely. If we talk about what can be found in storage systems most often, it is Infortrend and Mylex. CMD Technology and Digi-Data are also quite strong. For a long time Digi-Data has been the manufacturer of the fastest controllers on the mass market.
Of course, we should not forget that there are very strong solutions from vendors of complete storage solutions – Digital (now, part of Compaq), Andataco, Hitachi, Storage Computer and others, but they are of interest as complete systems, and that is the topic of another article.
Adaptec – SCSI fashion legislator and a leader in SCSI adapters, but in the RAID controller market it is not. In spite of this, the ARO series of RAID upgrade controllers is quite popular due to its exceptionally low price.
This series is represented by the ARO-1130CA, ARO-1130SA cards. They are designed for use on motherboards with Adaptec’s AIC-7880, AIC-7895 integrated SCSI chips and are to be installed on 32-bit PCI + RAIDport II connectors. The ARO-1130 CA supports RAID levels 0 and 1, and the ARO-1130SA supports RAID levels 0, 1 and 5. The disadvantages of using these solutions are their relatively low performance (these controllers do not use specialized processors to process I/O requests), a low number of features and a small set of drivers (Windows and Netware). Another problem with this series is incomplete BIOS compatibility with the motherboards that “support” them, so it is better to buy them from motherboard manufacturers who guarantee the compatibility of the products they sell.
AAA series controllers differ from ARO series by the presence of SCSI chips, so they can be used in motherboards without RAID port. AAA-131, AAA-133 cards (the difference is in the number of channels, respectively one and three) are supplied by Adaptec to distributor market, AAA-132 are available only for OEM.
Recently, Adaptec has introduced new RAID controllers supporting Ultra 2 SCSI interface. Among them are old series AAA and ARO (ARO-1130U2 can be used in motherboards with integrated Adaptec Ultra 2 SCSI controller and RAID port III, also UnixWare support appeared). As well as the new AAC-364 controllers, it is a 64 bit RAID controller with integrated powerful StrongARM® 233 MHz microprocessor and four Ultra 2 SCSI channels (two internal connectors and four external). It supports 128 M ECC cache memory and a Battery backup module, unlike its smaller counterparts. But its disadvantage is still the driver support only in Windows and Netware systems.
Adaptec also produces AEC-4312A and AEC-7312A series SCSI-to-SCSI controllers, they, like AAA series products, are rather simple and are used in entry level storage systems. Both models use AMD 5X86 processor, 133 MHz and support two Ultra SCSI disk channels each. The AEC-4312A model uses a single host channel with single ended or differential SCSI and Fast, or Ultra SCSI protocols, and the AEC-7312A model uses a single Fibre Channel host channel.
American Megatrends entered the RAID market in mid-1995. In 1996, AMI’s MegaRAID controller series outperformed its competitors in every way, and in early 1997, AMI became the leading supplier of PCI SCSI RAID controllers to OEMs. The OEM orientation for AMI is original, today more than 80% of world leading manufacturers use its controllers, but two years ago it entered the distributor market and its products became available through a network of resellers and distributors.
The main thing that AMI MegaRAID controllers have in common is high reliability, quality and the largest number of features compared to competitors’ products. When it comes to MegaRAID performance, they are almost always on the TPC Top Ten List and sometimes make up the vast majority.
Five models of AMI MegaRAID PCI RAID controllers are available today – 762, 466, 428, 434, 438.
The MegaRAID Express 762 series are Zero channel PCI RAID upgrade controllers for motherboards with integrated Symbios Logic SCSI controllers (including the Intel T440BX, NA440BX, NC440BX, SC450NX, AMI MegaRUM, MegaRUM II).
It supports RAID levels 0, 1, 3, 5, 10, 30, 50, JBOD (this also applies to other MegaRAID controllers) and can contain up to 128 MB of memory cache. Unlike Adaptec ARO controllers, the Express series can be installed on platforms with both conventional Ultra SCSI and Ultra2 SCSI (LVD).
The 466-series MegaRAID Express Plus controller differs from the 762-series controller in that, it can be installed in any motherboard, as it has an integrated SCSI chip Symbios Logic 53C895 (LVDS/SE). Both series have an integrated dedicated i960 I/O processor.
The disadvantages of the Express models are the inability to connect a BBU and the slower performance compared to full-featured devices.
Series MegaRAID 428 – is AMI’s classic full-featured RAID controller that combines high reliability, scalability and clustering support. It is available in single, dual and triple channel versions and supports up to 128MB of cache memory, which can be installed in two slots. The 428, 434 and 438 series, have BBU connectivity.
The 434 series differs from the 428 with a more powerful I/O processor and newer SCSI chips from Symbios Logic and no longer has built-in connectors for 50-pin cables. Unlike the 428 series, the 434 does not have clustering support, AMI decided to implement it in the next series, the 438.
The MegaRAID Ultra2 LVD series 438 only comes in two- and three-channel variants and has a number of innovations over its predecessor: Ultra2, I2O and clustering support. However, it only supports 64 MB of memory cache as a standard implementation.
Today it also comes with a version 438-H, which has better performance than the 438th controller thanks to a new specialized driver which, unfortunately, is designed only for Windows NT (in reality, except for the driver it does not differ from the usual 438-series).
From reliable sources recently reported that AMI is preparing to release several new products. Very soon there should be Express 300, Enterprise 1500, Enterprise 2000, and the first external Fiber-to-LVD RAID controller from AMI – Explorer 500. (At the time of writing, AMI was announcing the Explorer 500 and Enterprise 1500). All of the new products combine state-of-the-art technologies such as Ultra2 SCSI, SDRAM and will be several times faster than previous models.
Below is a brief summary of the features of the new MegaRAID models:
| Parameter | Express 300 | Enterprise 1500 | Enterprise 2000 | Explorer 500 |
| Processor | i960RM 100MHz | i960RN 100MHz | RISC 250MHz* | i960RN 100MHz |
| Кэш (SDRAM) | до 128M, 66MHz | до 128M, 66MHz | до 128M, 100MHz | до 128M, 66MHz |
| Host bus | 32 bit PCI | 64 bit PCI | 64 bit PCI | 2 x Fiber Channel |
| I2O | Yes | Yes | No | No |
| Clustering | Yes | Yes | Yes | Yes |
| Hot Plug PCI | No | No | Yes | No |
| BBU No | No | Yes | Yes | Yes |
Please note!
- 64bit 250MHz RISC Processor with AMI Companion Chip
- One Ultra2 SCSI channel on the controller + two channels on the motherboard
With a strong team of software developers, AMI has provided support for its controllers for all major operating systems. The standard package includes Novell Netware, Windows NT, SCO Unix, SCO Openserver, Unixware, Linux Redhat, Solaris, OS/2 Warp, MS DOS. By special request, support is provided for virtually any specialized system, there are also implementations for Banyan Vines and 64-bit Windows NT for ALPHA processors. In addition, the drivers for UNIX systems, including Solaris and Linux are the best performing among the competitors.
One of the problems of AMI is weak production capacity, as well as fairly high cost of products, but it should be noted that all products of this manufacturer is provided with 5-year warranty and have almost no defects.
DPT RAID controllers are known for their modality, performance and strong support of various operating systems. The list of supported operating systems includes almost all the most common ones – Novell Netware, Windows NT, SCO Unix, SCO Openserver, Unixware, Linux Redhat, OS/2 Warp, Win 95/98.
This year, DPT began shipping its fifth generation of RAID controllers – Smart RAID V. They are presented in three different series – Decade, Century and Millennium. SmartRAID V features DPT’s exclusive P3 (Parallel Pipeline Processing) technology, which allows you to process I/O commands in parallel and many other features convenient for array administration. In addition, DPT offers a fairly wide range of options for the use of Fiber Channel interface. Almost all models of new generation have variant of realization with this interface or can be extended with additional cards supporting Fiber Channel interface.
DPT controllers support RAID 0, 1, 0+1, 5, 0+5 and are among the most productive devices on the market. According to the Head of Sales and Marketing, the senior Millennium model is 4 times faster than the previous generation controllers, and is ahead of all competitors.
Millennium is positioned by DPT as a controller for enterprise servers. It has a very high performance (the central mechanism is implemented on i960HD-66MHz) and quite wide capabilities: it supports up to 3 Ultra2 SCSI or 2 Fiber channels, up to 256 MB of cache memory, is available in 32bit and 64bit PCI variants and has the possibility to connect the Battery Backup module (64bit variant only).
Century – are mid-range performance controllers at a competitive price. This model supports up to 64 MB of memory cache and is available for 32bit PCI in variants with 3 Ultra2 SCSI channels or one Fiber channel and one Ultra2 SCSI channel. Unlike Millennium, Century (as well as Decade) does not support BBU.
The Decade model is intended for use as an Entry Level RAID controller, is slower than the Century and is only available in a single channel Ultra2 SCSI variant. It supports 4 Mbytes of cache memory, which can be expanded to 64 Mbytes by installing a special expansion card.
As for the modularity – this is an exclusive feature of DPT RAID controllers, which allows the user, buying the basic solution, to expand it in the future using special expansion cards. They allow you to expand the capabilities of the used controller by adding one or two Ultra2 SCSI or one Fiber channel.
In addition to RAID controllers, DPT also produces a line of intelligent SCSI controllers with built-in I/O processor i960. When you connect a RAID Accelerator Module card to them, this controller becomes a RAID controller. Thus, the user saves on the fact that he invests his money in hardware as needed. It should also be noted that all SmartRAID V RAID controllers support I2O technology.
Mylex is the undisputed sales leader in the PCI SCSI RAID controller market. Last year Mylex had a market share of about 55%. Sales are conducted for both the OEM and distribution markets. Mylex PCI SCSI RAID controllers quite often top the TPC Top Ten List, proving their high performance.
Mylex categorizes the RAID controllers it manufactures into three groups.
- Low Cost RAID – a family of low-cost PCI RAID controllers, for lower-level servers and high-performance workstations. These include the AcceleRAID 150, 200 and 250 models. All of them are RAID upgrade controllers and can be used on motherboards with integrated Symbios Logic SCSI controller (like AMI MegaRAID Express series), but at the same time, 150 and 250 models have integrated SCSI chip and can be used in platforms without RAID port functions. They support RAID levels 0, 1, 0+1, 3, 5, 30, 50, JBOD, cache memory from 4 to 64 MB (AcceleRAID 150 – maximum 4 MB) and 32 bit Hot plug PCI.
- High Performance RAID – intelligent I/O solutions for mid- to high-end servers with high fault tolerance and easy management. These include the eXtremeRAID 1100, DAC960PJ / DAC960PG controllers. They have a fairly large number of opportunities to administer RAID and provide performance at the level required for enterprise servers.
The PJ and PG models do not differ much, except that the first is integrated more powerful processor – Intel i960RD in contrast to the Intel i960RP, which is integrated in the PG. The eXtremeRAID 1100 is the first implementation of the Mylex RAID controller for 64bit PCI (although its internal architecture remains 32-bit).
The eXtremeRAID 1100 has a StrongArm SA 110, 233MHz processor integrated to handle I/O requests, making it one of the most powerful PCI-to-SCSI controllers available today. Along with these benefits, the eXtremeRAID 1100 is certified for cluster systems running Windows NT Enterprise Edition.
Mylex has provided support for its PCI-to-SCSI controllers for the most common server operating systems such as: Novell Netware, Windows NT, SCO Unix, SCO Openserver, Unixware, Linux Redhat, Win 95/98.
The disadvantage of Mylex PCI-to-SCSI solutions is a relatively low MTBF of 200,000 hours, while most competitor models have this figure ranging from 350,000 to 500,000 hours.
- External RAID Mylex solutions are high-speed and optimally flexible solutions for the most demanding enterprise and mid-range server systems. They are ideal for Storage Area Networking and server clustering.
The Mylex line of external RAID controllers includes 5 basic models (we will not consider the SU model as obsolete).
| Parameter | DAC960SX | DAC960SF | DAC960FL | DAC960FF |
| CPU (i960) | RD 33MHz | 2 x RD 66MHz | 2 x RD 66MHz | RN 100MHz |
| Cache | 128Mb | 128Mb | 256Mb | 256Mb |
| Host bus | 1(2) Ultra SCSI | 2 x FC | 2 x FC | 2 x FC |
| Drive bus | 2(5) Ultra SCSI | 4(6) Ultra SCSI | 4 x Ultra2 SCSI | 4 x FС |
| Burst IO | 1’800 | 4’100 | 4’100 | 5’800 |
| Transfer | 30 MB/s | 52 MB/s | 52 MB/s | 190 MB/s |
| Disks | 75 | 90 | 60 | 500 |
Please note!
- Minimum and maximum (in parentheses) number of host channels for the model.
- Minimum and maximum (in parentheses) number of disk channels for the model.
- Maximum I/O processing speed.
- Continuous disk transfer rate, is actually the maximum continuous transfer rate available, while the peak rate can reach the maximum possible value for the total throughput of the host channels.
- Maximum number of disks connected to a single controller.
IFT (Infortrend technology) – has been in the RAID market since 1992. It was organized as a company for the development and production of high-reliability and high-performance controllers. Thanks to interesting ideas and low prices it managed to win quite a big share of the OEM market. Its customers include companies like Amaquest (http://www.amaqest.com.tw/) and ASUStek (http://www.asus.com.tw/), among many others.
IFT makes a single model of PCI-to-SCSI controllers 2101UA/B (A – 1 channel, B – 2 channels) which has recently been extended with an implementation supporting Ultra 2 (which also uses a more powerful processor). They use x86 series processors for I/O (2101U – 486DX4 100MHz; 2101U2 – AMD 5×86 133 MHz), which makes them quite attractive for entry level systems, due to good price/performance ratio. IFT PCI-to-SCSI controllers have very good driver support, there are drivers for them written for Novell Netware, Windows NT, Win95/98, OS/2 Warp, SCO Unix, SCO Openserver, Unixware, Linux, SUN Solaris.
As for subsystem-based RAID controllers, the IFT is doing much better. The Infortrend SCSI-to-SCSI product line includes a large number of models ranging from simple 2-channel in 3.5″ form factor to 9-channel models with fiber optic host channels. Although IFT RAID controllers are not industry benchmarks for performance, they combine high reliability with an excellent price/performance ratio and have earned their place in entry level and mid-range systems.
Models Series 3101 (3.5 “entry level) are simple enough, have a few channels (from 2 to 4), and like all external controllers of this manufacturer are Multihost. The 3102 series (5.25”, mid-range) is more robust and easy to integrate, according to the manufacturer, and also has a wider range of expansion capabilities.
The IFT controllers are modular. The 3102 series includes four models, the 3102U and 3102UG, and the newer 3102U2 and 3102U2G. All of these models use SCSI channels in the basic version and are Multihost controllers (unlike the classic Multihost implementation which assumes that any channel can be used for host connection, or for disk connection).
| Parameter | 3101U2G | 3102U | 3102UG | 3102U2 | 3102U2G |
| CPU | 5×86 133MHz | 486 100MHz | 486 100MHz | 5×86 133MHz | 5×86 133MHz |
| Cache, to | 128Mb | 128Mb | 128Mb | 128Mb | 128Mb |
| Interface | Ultra2 SCSI | Ultra SCSI | Ultra SCSI | Ultra2 SCSI | Ultra2 SCSI |
| Channels | 2 channels | 3 channels | 4 channels | 3 channels | 4 channels |
| Expandable | 4 channels | 8 channels | 9 channels | 6 channels | 8 channels |
| BBU | No | Yes | Yes | Yes | Yes |
| HotSwap | Yes | No | Yes | No | Yes |
Please note!
- Channel interface of the base module;
- Each channel can be used for host communication or drive connection;
- Maximum number of channels after ramping.
Each model can be expanded using daughter boards. The set of such boards for each model of the same series is very similar. As an example, let’s look at the expansion options of the most modern model 3102U2G.
- IFT-9174: 4 x 68-pin Ultra2 Wide SCSI (with termination);
- IFT-9174-N: 4 x 68-pin Ultra2 Wide SCSI (no termination);
- IFT-9174U2D: 2 x 68-pin Ultra2 Wide SCSI and 2 x 68-pin Ultra Wide SCSI (with termination);
- IFT-9174U2D-N: 2 x 68-pin Ultra2 Wide SCSI and 2 x 68-pin Ultra Wide differential (no termination);
- IFT-9174U2F: 2 x 68-pin Ultra Wide, single ended & 2 x single loop Fibre channels* (with termination);
- IFT-9174U2F-N: 2 x 68-pin Ultra2 Wide SCSI and 2 x single loop Fibre channels (no termination).
Please note!
- Two single loop fiber channels can be used as one dual loop channel.
The disadvantage of Infortrend still remains the lack of high-performance controllers, designed for high rates of constant continuous transfer.
CMD Technology, in contrast to the above companies, produces RAID controllers only in the form of standalone subsystems (SCSI-to-SCSI RAID). This American manufacturer has long been a supplier of external RAID controllers for Digital storage systems, which undoubtedly characterizes it as a manufacturer of high-quality, reliable and fast devices.
The CMD product line includes models targeted at OEMs, resellers and system integrators for entry-level, mid-level and high-end storage applications. We take a brief look at models designed for use by resellers and system integrators (including small OEMs).
The Viper II series includes the CRD-5440, CRD5500, and CRD-564X models. It supports RAID levels 0, 1, 0+1, 4, 5. The CRD-5440 model is designed for use by integrators and resellers in entry level and mid-range systems. The CRD-5440 controllers use four SCSI channels (interface: Ultra SCSI Low Voltage Differential (LVD), Single Ended (SE) or High Voltage Differential (LVD)), which can be used for both host connection and drive connection. A 32-bit RISC 40 Mhz LR33310 (MIPS R3000 core) processor with an internal transfer ratio of 80Mb/s and supports a cache memory up to 256Mb is used to process I/O commands.
The CRD-5500 is targeted at high-speed, high-reliability and fault-tolerant storage systems. The architecture of the CRD5500 is organized as an Active/Active fault tolerant modular controller based on the same processor as the CRD-5400, but unlike its competitors, it is built in such a way that in Active/Active configuration the data transfer ratio is quite high and is approximately 1.7 of nominal. The CRD5500 is modular and can be configured with up to four host and eight drive connection channels and offers up to 512MB cache memory.
The CRD-564X is designed to provide the highest level of data integrity and availability in entry-to-midrange systems. The CRD-564X is a fault-tolerant, off-the-shelf Viper II RAID controller. Like all other devices in this series, the CRD-564X uses a 40 Mhz LR 33310 I/O processor and supports RAID 0, 1, 0+1, 4, 5, and supports four Ultra SCSI channels.
An important feature is support for CMD Auto Rebuild technology. If a single controller fails (the CRD-564X consists of two controllers connected one to one in the same chassis) and is replaced, this technology automatically returns the failover controller (meaning the entire CRD-564X unit) to the state it was in before the failure, without user intervention. In addition, every part of the controller is a hot pluggable device (hot swappable).
Titan CRA-7280 – the latest series of RAID controllers. The Titan architecture combines Fibre Channel and LVD Ultra2 SCSI technologies. This series is focused on RAID systems for Storage Area Network (SAN) applications.
The CRA-7280 is a high-speed, high-reliability, redundant controller for mid- to high-end systems, it features CMD Auto Rebuild technology, and comes in a 3U 19″ Rackmount chassis for easy integration into a 19″ rack.
The Titan controller architecture is based on a 233 MHz SA-110 StrongARM RISC CPU with an internal 32-bit and 64-bit data bus and SDRAM support. The CRA-7280 uses two Fibre Channel Arbitrated Loops (FCAL) host interfaces in copper, single-mode or multi-mode fiber implementations and supports up to a 1Gb memory cache.
Digi-Data – American manufacturer of SCSI-to-SCSI RAID controllers, which for a long time were the fastest among the devices of their class. Digi-Data has been working in the storage market since 1960. The company began producing RAID controllers in 1992, and has since earned a reputation as a manufacturer of high-quality, reliable devices.
Today, Digi-Data’s line of controllers includes the series:
- Z-9100 – Ultra SCSI RAID controllers;
- Z-9200 – Ultra2 SCSI RAID controllers;
- Z-9500 – Fibre Channel RAID controllers.
Digi-Data controllers are perfect for handling large data sets and continuous data streams for applications such as video or satellite stream logging.
- The Z-9100 series includes four models: Z-9100, Z-9102, Z-9150, Z-9152. All of them use six disk channels (four for data, one for parity, one for spare disks) and support RAID levels 0, 1 and 3. The models differ in having one or two (the number 2 at the end of the model number) host interfaces and support RAID level 5;
- The Z-9200 series consists of two models: Z-9200, Z-9250. They differ only in their support for level 5 RAID. It should also be noted that the Z-9200 series is much faster than the Z-9100. With only five Seagate Cheetah 9LP disks they achieve over 50MB/s continuous transfer, with more disks up to 60MB/s;
- The Z-9500 series differs from the Z-9100 in that it uses Fiber Channel instead of Ultra SCSI interfaces for communication with the host machines, as well as in the increased speed. It should also be noted that all models of Digi-Data RAID controllers support up to 256 MB of memory cache.
Digi-Data products feature a number of proprietary technologies to ensure high performance and fault tolerance. Among the most interesting and useful are:
- Full-Speed Active-Active with FASTCORE is a technology that provides increased performance of two RAID controllers combined in an Active/Active configuration. Thanks to this technology and the use of a special additional device – repeater, achieved twice the performance than when using a single controller;
- Self Calibrating Automatic Tier Striping (SCATS) is a software technology that implements two spatial separation of data streams with optimized subblock sizes;
- UVS (Uninterruptible Video Streaming) – technology that provides continuous data transfer in case of failure (many RAID controllers do not do this, in case of failure of a disk or cluster usually stalls);
- Guaranteed RAID Sustained Data Transfer Rate – Digi-Data guarantees high data transfer rates for all supported levels and for various operations (both read and write).
Conclusion of the review on RAID controllers
Remember, when choosing RAID controllers do not rely only on performance figures (the real difference between the performance of similar models is usually visible on a large number of disks), the main thing to determine the key points – speed, reliability, stability, scalability, clustering support, price – and choose exactly what best meets the necessary requirements.
