Back to home page

OSCL-LXR

 
 

    


0001 ==============
0002 Control Groups
0003 ==============
0004 
0005 Written by Paul Menage <menage@google.com> based on
0006 Documentation/admin-guide/cgroup-v1/cpusets.rst
0007 
0008 Original copyright statements from cpusets.txt:
0009 
0010 Portions Copyright (C) 2004 BULL SA.
0011 
0012 Portions Copyright (c) 2004-2006 Silicon Graphics, Inc.
0013 
0014 Modified by Paul Jackson <pj@sgi.com>
0015 
0016 Modified by Christoph Lameter <cl@linux.com>
0017 
0018 .. CONTENTS:
0019 
0020         1. Control Groups
0021         1.1 What are cgroups ?
0022         1.2 Why are cgroups needed ?
0023         1.3 How are cgroups implemented ?
0024         1.4 What does notify_on_release do ?
0025         1.5 What does clone_children do ?
0026         1.6 How do I use cgroups ?
0027         2. Usage Examples and Syntax
0028         2.1 Basic Usage
0029         2.2 Attaching processes
0030         2.3 Mounting hierarchies by name
0031         3. Kernel API
0032         3.1 Overview
0033         3.2 Synchronization
0034         3.3 Subsystem API
0035         4. Extended attributes usage
0036         5. Questions
0037 
0038 1. Control Groups
0039 =================
0040 
0041 1.1 What are cgroups ?
0042 ----------------------
0043 
0044 Control Groups provide a mechanism for aggregating/partitioning sets of
0045 tasks, and all their future children, into hierarchical groups with
0046 specialized behaviour.
0047 
0048 Definitions:
0049 
0050 A *cgroup* associates a set of tasks with a set of parameters for one
0051 or more subsystems.
0052 
0053 A *subsystem* is a module that makes use of the task grouping
0054 facilities provided by cgroups to treat groups of tasks in
0055 particular ways. A subsystem is typically a "resource controller" that
0056 schedules a resource or applies per-cgroup limits, but it may be
0057 anything that wants to act on a group of processes, e.g. a
0058 virtualization subsystem.
0059 
0060 A *hierarchy* is a set of cgroups arranged in a tree, such that
0061 every task in the system is in exactly one of the cgroups in the
0062 hierarchy, and a set of subsystems; each subsystem has system-specific
0063 state attached to each cgroup in the hierarchy.  Each hierarchy has
0064 an instance of the cgroup virtual filesystem associated with it.
0065 
0066 At any one time there may be multiple active hierarchies of task
0067 cgroups. Each hierarchy is a partition of all tasks in the system.
0068 
0069 User-level code may create and destroy cgroups by name in an
0070 instance of the cgroup virtual file system, specify and query to
0071 which cgroup a task is assigned, and list the task PIDs assigned to
0072 a cgroup. Those creations and assignments only affect the hierarchy
0073 associated with that instance of the cgroup file system.
0074 
0075 On their own, the only use for cgroups is for simple job
0076 tracking. The intention is that other subsystems hook into the generic
0077 cgroup support to provide new attributes for cgroups, such as
0078 accounting/limiting the resources which processes in a cgroup can
0079 access. For example, cpusets (see Documentation/admin-guide/cgroup-v1/cpusets.rst) allow
0080 you to associate a set of CPUs and a set of memory nodes with the
0081 tasks in each cgroup.
0082 
0083 1.2 Why are cgroups needed ?
0084 ----------------------------
0085 
0086 There are multiple efforts to provide process aggregations in the
0087 Linux kernel, mainly for resource-tracking purposes. Such efforts
0088 include cpusets, CKRM/ResGroups, UserBeanCounters, and virtual server
0089 namespaces. These all require the basic notion of a
0090 grouping/partitioning of processes, with newly forked processes ending
0091 up in the same group (cgroup) as their parent process.
0092 
0093 The kernel cgroup patch provides the minimum essential kernel
0094 mechanisms required to efficiently implement such groups. It has
0095 minimal impact on the system fast paths, and provides hooks for
0096 specific subsystems such as cpusets to provide additional behaviour as
0097 desired.
0098 
0099 Multiple hierarchy support is provided to allow for situations where
0100 the division of tasks into cgroups is distinctly different for
0101 different subsystems - having parallel hierarchies allows each
0102 hierarchy to be a natural division of tasks, without having to handle
0103 complex combinations of tasks that would be present if several
0104 unrelated subsystems needed to be forced into the same tree of
0105 cgroups.
0106 
0107 At one extreme, each resource controller or subsystem could be in a
0108 separate hierarchy; at the other extreme, all subsystems
0109 would be attached to the same hierarchy.
0110 
0111 As an example of a scenario (originally proposed by vatsa@in.ibm.com)
0112 that can benefit from multiple hierarchies, consider a large
0113 university server with various users - students, professors, system
0114 tasks etc. The resource planning for this server could be along the
0115 following lines::
0116 
0117        CPU :          "Top cpuset"
0118                        /       \
0119                CPUSet1         CPUSet2
0120                   |               |
0121                (Professors)    (Students)
0122 
0123                In addition (system tasks) are attached to topcpuset (so
0124                that they can run anywhere) with a limit of 20%
0125 
0126        Memory : Professors (50%), Students (30%), system (20%)
0127 
0128        Disk : Professors (50%), Students (30%), system (20%)
0129 
0130        Network : WWW browsing (20%), Network File System (60%), others (20%)
0131                                / \
0132                Professors (15%)  students (5%)
0133 
0134 Browsers like Firefox/Lynx go into the WWW network class, while (k)nfsd goes
0135 into the NFS network class.
0136 
0137 At the same time Firefox/Lynx will share an appropriate CPU/Memory class
0138 depending on who launched it (prof/student).
0139 
0140 With the ability to classify tasks differently for different resources
0141 (by putting those resource subsystems in different hierarchies),
0142 the admin can easily set up a script which receives exec notifications
0143 and depending on who is launching the browser he can::
0144 
0145     # echo browser_pid > /sys/fs/cgroup/<restype>/<userclass>/tasks
0146 
0147 With only a single hierarchy, he now would potentially have to create
0148 a separate cgroup for every browser launched and associate it with
0149 appropriate network and other resource class.  This may lead to
0150 proliferation of such cgroups.
0151 
0152 Also let's say that the administrator would like to give enhanced network
0153 access temporarily to a student's browser (since it is night and the user
0154 wants to do online gaming :))  OR give one of the student's simulation
0155 apps enhanced CPU power.
0156 
0157 With ability to write PIDs directly to resource classes, it's just a
0158 matter of::
0159 
0160        # echo pid > /sys/fs/cgroup/network/<new_class>/tasks
0161        (after some time)
0162        # echo pid > /sys/fs/cgroup/network/<orig_class>/tasks
0163 
0164 Without this ability, the administrator would have to split the cgroup into
0165 multiple separate ones and then associate the new cgroups with the
0166 new resource classes.
0167 
0168 
0169 
0170 1.3 How are cgroups implemented ?
0171 ---------------------------------
0172 
0173 Control Groups extends the kernel as follows:
0174 
0175  - Each task in the system has a reference-counted pointer to a
0176    css_set.
0177 
0178  - A css_set contains a set of reference-counted pointers to
0179    cgroup_subsys_state objects, one for each cgroup subsystem
0180    registered in the system. There is no direct link from a task to
0181    the cgroup of which it's a member in each hierarchy, but this
0182    can be determined by following pointers through the
0183    cgroup_subsys_state objects. This is because accessing the
0184    subsystem state is something that's expected to happen frequently
0185    and in performance-critical code, whereas operations that require a
0186    task's actual cgroup assignments (in particular, moving between
0187    cgroups) are less common. A linked list runs through the cg_list
0188    field of each task_struct using the css_set, anchored at
0189    css_set->tasks.
0190 
0191  - A cgroup hierarchy filesystem can be mounted for browsing and
0192    manipulation from user space.
0193 
0194  - You can list all the tasks (by PID) attached to any cgroup.
0195 
0196 The implementation of cgroups requires a few, simple hooks
0197 into the rest of the kernel, none in performance-critical paths:
0198 
0199  - in init/main.c, to initialize the root cgroups and initial
0200    css_set at system boot.
0201 
0202  - in fork and exit, to attach and detach a task from its css_set.
0203 
0204 In addition, a new file system of type "cgroup" may be mounted, to
0205 enable browsing and modifying the cgroups presently known to the
0206 kernel.  When mounting a cgroup hierarchy, you may specify a
0207 comma-separated list of subsystems to mount as the filesystem mount
0208 options.  By default, mounting the cgroup filesystem attempts to
0209 mount a hierarchy containing all registered subsystems.
0210 
0211 If an active hierarchy with exactly the same set of subsystems already
0212 exists, it will be reused for the new mount. If no existing hierarchy
0213 matches, and any of the requested subsystems are in use in an existing
0214 hierarchy, the mount will fail with -EBUSY. Otherwise, a new hierarchy
0215 is activated, associated with the requested subsystems.
0216 
0217 It's not currently possible to bind a new subsystem to an active
0218 cgroup hierarchy, or to unbind a subsystem from an active cgroup
0219 hierarchy. This may be possible in future, but is fraught with nasty
0220 error-recovery issues.
0221 
0222 When a cgroup filesystem is unmounted, if there are any
0223 child cgroups created below the top-level cgroup, that hierarchy
0224 will remain active even though unmounted; if there are no
0225 child cgroups then the hierarchy will be deactivated.
0226 
0227 No new system calls are added for cgroups - all support for
0228 querying and modifying cgroups is via this cgroup file system.
0229 
0230 Each task under /proc has an added file named 'cgroup' displaying,
0231 for each active hierarchy, the subsystem names and the cgroup name
0232 as the path relative to the root of the cgroup file system.
0233 
0234 Each cgroup is represented by a directory in the cgroup file system
0235 containing the following files describing that cgroup:
0236 
0237  - tasks: list of tasks (by PID) attached to that cgroup.  This list
0238    is not guaranteed to be sorted.  Writing a thread ID into this file
0239    moves the thread into this cgroup.
0240  - cgroup.procs: list of thread group IDs in the cgroup.  This list is
0241    not guaranteed to be sorted or free of duplicate TGIDs, and userspace
0242    should sort/uniquify the list if this property is required.
0243    Writing a thread group ID into this file moves all threads in that
0244    group into this cgroup.
0245  - notify_on_release flag: run the release agent on exit?
0246  - release_agent: the path to use for release notifications (this file
0247    exists in the top cgroup only)
0248 
0249 Other subsystems such as cpusets may add additional files in each
0250 cgroup dir.
0251 
0252 New cgroups are created using the mkdir system call or shell
0253 command.  The properties of a cgroup, such as its flags, are
0254 modified by writing to the appropriate file in that cgroups
0255 directory, as listed above.
0256 
0257 The named hierarchical structure of nested cgroups allows partitioning
0258 a large system into nested, dynamically changeable, "soft-partitions".
0259 
0260 The attachment of each task, automatically inherited at fork by any
0261 children of that task, to a cgroup allows organizing the work load
0262 on a system into related sets of tasks.  A task may be re-attached to
0263 any other cgroup, if allowed by the permissions on the necessary
0264 cgroup file system directories.
0265 
0266 When a task is moved from one cgroup to another, it gets a new
0267 css_set pointer - if there's an already existing css_set with the
0268 desired collection of cgroups then that group is reused, otherwise a new
0269 css_set is allocated. The appropriate existing css_set is located by
0270 looking into a hash table.
0271 
0272 To allow access from a cgroup to the css_sets (and hence tasks)
0273 that comprise it, a set of cg_cgroup_link objects form a lattice;
0274 each cg_cgroup_link is linked into a list of cg_cgroup_links for
0275 a single cgroup on its cgrp_link_list field, and a list of
0276 cg_cgroup_links for a single css_set on its cg_link_list.
0277 
0278 Thus the set of tasks in a cgroup can be listed by iterating over
0279 each css_set that references the cgroup, and sub-iterating over
0280 each css_set's task set.
0281 
0282 The use of a Linux virtual file system (vfs) to represent the
0283 cgroup hierarchy provides for a familiar permission and name space
0284 for cgroups, with a minimum of additional kernel code.
0285 
0286 1.4 What does notify_on_release do ?
0287 ------------------------------------
0288 
0289 If the notify_on_release flag is enabled (1) in a cgroup, then
0290 whenever the last task in the cgroup leaves (exits or attaches to
0291 some other cgroup) and the last child cgroup of that cgroup
0292 is removed, then the kernel runs the command specified by the contents
0293 of the "release_agent" file in that hierarchy's root directory,
0294 supplying the pathname (relative to the mount point of the cgroup
0295 file system) of the abandoned cgroup.  This enables automatic
0296 removal of abandoned cgroups.  The default value of
0297 notify_on_release in the root cgroup at system boot is disabled
0298 (0).  The default value of other cgroups at creation is the current
0299 value of their parents' notify_on_release settings. The default value of
0300 a cgroup hierarchy's release_agent path is empty.
0301 
0302 1.5 What does clone_children do ?
0303 ---------------------------------
0304 
0305 This flag only affects the cpuset controller. If the clone_children
0306 flag is enabled (1) in a cgroup, a new cpuset cgroup will copy its
0307 configuration from the parent during initialization.
0308 
0309 1.6 How do I use cgroups ?
0310 --------------------------
0311 
0312 To start a new job that is to be contained within a cgroup, using
0313 the "cpuset" cgroup subsystem, the steps are something like::
0314 
0315  1) mount -t tmpfs cgroup_root /sys/fs/cgroup
0316  2) mkdir /sys/fs/cgroup/cpuset
0317  3) mount -t cgroup -ocpuset cpuset /sys/fs/cgroup/cpuset
0318  4) Create the new cgroup by doing mkdir's and write's (or echo's) in
0319     the /sys/fs/cgroup/cpuset virtual file system.
0320  5) Start a task that will be the "founding father" of the new job.
0321  6) Attach that task to the new cgroup by writing its PID to the
0322     /sys/fs/cgroup/cpuset tasks file for that cgroup.
0323  7) fork, exec or clone the job tasks from this founding father task.
0324 
0325 For example, the following sequence of commands will setup a cgroup
0326 named "Charlie", containing just CPUs 2 and 3, and Memory Node 1,
0327 and then start a subshell 'sh' in that cgroup::
0328 
0329   mount -t tmpfs cgroup_root /sys/fs/cgroup
0330   mkdir /sys/fs/cgroup/cpuset
0331   mount -t cgroup cpuset -ocpuset /sys/fs/cgroup/cpuset
0332   cd /sys/fs/cgroup/cpuset
0333   mkdir Charlie
0334   cd Charlie
0335   /bin/echo 2-3 > cpuset.cpus
0336   /bin/echo 1 > cpuset.mems
0337   /bin/echo $$ > tasks
0338   sh
0339   # The subshell 'sh' is now running in cgroup Charlie
0340   # The next line should display '/Charlie'
0341   cat /proc/self/cgroup
0342 
0343 2. Usage Examples and Syntax
0344 ============================
0345 
0346 2.1 Basic Usage
0347 ---------------
0348 
0349 Creating, modifying, using cgroups can be done through the cgroup
0350 virtual filesystem.
0351 
0352 To mount a cgroup hierarchy with all available subsystems, type::
0353 
0354   # mount -t cgroup xxx /sys/fs/cgroup
0355 
0356 The "xxx" is not interpreted by the cgroup code, but will appear in
0357 /proc/mounts so may be any useful identifying string that you like.
0358 
0359 Note: Some subsystems do not work without some user input first.  For instance,
0360 if cpusets are enabled the user will have to populate the cpus and mems files
0361 for each new cgroup created before that group can be used.
0362 
0363 As explained in section `1.2 Why are cgroups needed?` you should create
0364 different hierarchies of cgroups for each single resource or group of
0365 resources you want to control. Therefore, you should mount a tmpfs on
0366 /sys/fs/cgroup and create directories for each cgroup resource or resource
0367 group::
0368 
0369   # mount -t tmpfs cgroup_root /sys/fs/cgroup
0370   # mkdir /sys/fs/cgroup/rg1
0371 
0372 To mount a cgroup hierarchy with just the cpuset and memory
0373 subsystems, type::
0374 
0375   # mount -t cgroup -o cpuset,memory hier1 /sys/fs/cgroup/rg1
0376 
0377 While remounting cgroups is currently supported, it is not recommend
0378 to use it. Remounting allows changing bound subsystems and
0379 release_agent. Rebinding is hardly useful as it only works when the
0380 hierarchy is empty and release_agent itself should be replaced with
0381 conventional fsnotify. The support for remounting will be removed in
0382 the future.
0383 
0384 To Specify a hierarchy's release_agent::
0385 
0386   # mount -t cgroup -o cpuset,release_agent="/sbin/cpuset_release_agent" \
0387     xxx /sys/fs/cgroup/rg1
0388 
0389 Note that specifying 'release_agent' more than once will return failure.
0390 
0391 Note that changing the set of subsystems is currently only supported
0392 when the hierarchy consists of a single (root) cgroup. Supporting
0393 the ability to arbitrarily bind/unbind subsystems from an existing
0394 cgroup hierarchy is intended to be implemented in the future.
0395 
0396 Then under /sys/fs/cgroup/rg1 you can find a tree that corresponds to the
0397 tree of the cgroups in the system. For instance, /sys/fs/cgroup/rg1
0398 is the cgroup that holds the whole system.
0399 
0400 If you want to change the value of release_agent::
0401 
0402   # echo "/sbin/new_release_agent" > /sys/fs/cgroup/rg1/release_agent
0403 
0404 It can also be changed via remount.
0405 
0406 If you want to create a new cgroup under /sys/fs/cgroup/rg1::
0407 
0408   # cd /sys/fs/cgroup/rg1
0409   # mkdir my_cgroup
0410 
0411 Now you want to do something with this cgroup:
0412 
0413   # cd my_cgroup
0414 
0415 In this directory you can find several files::
0416 
0417   # ls
0418   cgroup.procs notify_on_release tasks
0419   (plus whatever files added by the attached subsystems)
0420 
0421 Now attach your shell to this cgroup::
0422 
0423   # /bin/echo $$ > tasks
0424 
0425 You can also create cgroups inside your cgroup by using mkdir in this
0426 directory::
0427 
0428   # mkdir my_sub_cs
0429 
0430 To remove a cgroup, just use rmdir::
0431 
0432   # rmdir my_sub_cs
0433 
0434 This will fail if the cgroup is in use (has cgroups inside, or
0435 has processes attached, or is held alive by other subsystem-specific
0436 reference).
0437 
0438 2.2 Attaching processes
0439 -----------------------
0440 
0441 ::
0442 
0443   # /bin/echo PID > tasks
0444 
0445 Note that it is PID, not PIDs. You can only attach ONE task at a time.
0446 If you have several tasks to attach, you have to do it one after another::
0447 
0448   # /bin/echo PID1 > tasks
0449   # /bin/echo PID2 > tasks
0450           ...
0451   # /bin/echo PIDn > tasks
0452 
0453 You can attach the current shell task by echoing 0::
0454 
0455   # echo 0 > tasks
0456 
0457 You can use the cgroup.procs file instead of the tasks file to move all
0458 threads in a threadgroup at once. Echoing the PID of any task in a
0459 threadgroup to cgroup.procs causes all tasks in that threadgroup to be
0460 attached to the cgroup. Writing 0 to cgroup.procs moves all tasks
0461 in the writing task's threadgroup.
0462 
0463 Note: Since every task is always a member of exactly one cgroup in each
0464 mounted hierarchy, to remove a task from its current cgroup you must
0465 move it into a new cgroup (possibly the root cgroup) by writing to the
0466 new cgroup's tasks file.
0467 
0468 Note: Due to some restrictions enforced by some cgroup subsystems, moving
0469 a process to another cgroup can fail.
0470 
0471 2.3 Mounting hierarchies by name
0472 --------------------------------
0473 
0474 Passing the name=<x> option when mounting a cgroups hierarchy
0475 associates the given name with the hierarchy.  This can be used when
0476 mounting a pre-existing hierarchy, in order to refer to it by name
0477 rather than by its set of active subsystems.  Each hierarchy is either
0478 nameless, or has a unique name.
0479 
0480 The name should match [\w.-]+
0481 
0482 When passing a name=<x> option for a new hierarchy, you need to
0483 specify subsystems manually; the legacy behaviour of mounting all
0484 subsystems when none are explicitly specified is not supported when
0485 you give a subsystem a name.
0486 
0487 The name of the subsystem appears as part of the hierarchy description
0488 in /proc/mounts and /proc/<pid>/cgroups.
0489 
0490 
0491 3. Kernel API
0492 =============
0493 
0494 3.1 Overview
0495 ------------
0496 
0497 Each kernel subsystem that wants to hook into the generic cgroup
0498 system needs to create a cgroup_subsys object. This contains
0499 various methods, which are callbacks from the cgroup system, along
0500 with a subsystem ID which will be assigned by the cgroup system.
0501 
0502 Other fields in the cgroup_subsys object include:
0503 
0504 - subsys_id: a unique array index for the subsystem, indicating which
0505   entry in cgroup->subsys[] this subsystem should be managing.
0506 
0507 - name: should be initialized to a unique subsystem name. Should be
0508   no longer than MAX_CGROUP_TYPE_NAMELEN.
0509 
0510 - early_init: indicate if the subsystem needs early initialization
0511   at system boot.
0512 
0513 Each cgroup object created by the system has an array of pointers,
0514 indexed by subsystem ID; this pointer is entirely managed by the
0515 subsystem; the generic cgroup code will never touch this pointer.
0516 
0517 3.2 Synchronization
0518 -------------------
0519 
0520 There is a global mutex, cgroup_mutex, used by the cgroup
0521 system. This should be taken by anything that wants to modify a
0522 cgroup. It may also be taken to prevent cgroups from being
0523 modified, but more specific locks may be more appropriate in that
0524 situation.
0525 
0526 See kernel/cgroup.c for more details.
0527 
0528 Subsystems can take/release the cgroup_mutex via the functions
0529 cgroup_lock()/cgroup_unlock().
0530 
0531 Accessing a task's cgroup pointer may be done in the following ways:
0532 - while holding cgroup_mutex
0533 - while holding the task's alloc_lock (via task_lock())
0534 - inside an rcu_read_lock() section via rcu_dereference()
0535 
0536 3.3 Subsystem API
0537 -----------------
0538 
0539 Each subsystem should:
0540 
0541 - add an entry in linux/cgroup_subsys.h
0542 - define a cgroup_subsys object called <name>_cgrp_subsys
0543 
0544 Each subsystem may export the following methods. The only mandatory
0545 methods are css_alloc/free. Any others that are null are presumed to
0546 be successful no-ops.
0547 
0548 ``struct cgroup_subsys_state *css_alloc(struct cgroup *cgrp)``
0549 (cgroup_mutex held by caller)
0550 
0551 Called to allocate a subsystem state object for a cgroup. The
0552 subsystem should allocate its subsystem state object for the passed
0553 cgroup, returning a pointer to the new object on success or a
0554 ERR_PTR() value. On success, the subsystem pointer should point to
0555 a structure of type cgroup_subsys_state (typically embedded in a
0556 larger subsystem-specific object), which will be initialized by the
0557 cgroup system. Note that this will be called at initialization to
0558 create the root subsystem state for this subsystem; this case can be
0559 identified by the passed cgroup object having a NULL parent (since
0560 it's the root of the hierarchy) and may be an appropriate place for
0561 initialization code.
0562 
0563 ``int css_online(struct cgroup *cgrp)``
0564 (cgroup_mutex held by caller)
0565 
0566 Called after @cgrp successfully completed all allocations and made
0567 visible to cgroup_for_each_child/descendant_*() iterators. The
0568 subsystem may choose to fail creation by returning -errno. This
0569 callback can be used to implement reliable state sharing and
0570 propagation along the hierarchy. See the comment on
0571 cgroup_for_each_descendant_pre() for details.
0572 
0573 ``void css_offline(struct cgroup *cgrp);``
0574 (cgroup_mutex held by caller)
0575 
0576 This is the counterpart of css_online() and called iff css_online()
0577 has succeeded on @cgrp. This signifies the beginning of the end of
0578 @cgrp. @cgrp is being removed and the subsystem should start dropping
0579 all references it's holding on @cgrp. When all references are dropped,
0580 cgroup removal will proceed to the next step - css_free(). After this
0581 callback, @cgrp should be considered dead to the subsystem.
0582 
0583 ``void css_free(struct cgroup *cgrp)``
0584 (cgroup_mutex held by caller)
0585 
0586 The cgroup system is about to free @cgrp; the subsystem should free
0587 its subsystem state object. By the time this method is called, @cgrp
0588 is completely unused; @cgrp->parent is still valid. (Note - can also
0589 be called for a newly-created cgroup if an error occurs after this
0590 subsystem's create() method has been called for the new cgroup).
0591 
0592 ``int can_attach(struct cgroup *cgrp, struct cgroup_taskset *tset)``
0593 (cgroup_mutex held by caller)
0594 
0595 Called prior to moving one or more tasks into a cgroup; if the
0596 subsystem returns an error, this will abort the attach operation.
0597 @tset contains the tasks to be attached and is guaranteed to have at
0598 least one task in it.
0599 
0600 If there are multiple tasks in the taskset, then:
0601   - it's guaranteed that all are from the same thread group
0602   - @tset contains all tasks from the thread group whether or not
0603     they're switching cgroups
0604   - the first task is the leader
0605 
0606 Each @tset entry also contains the task's old cgroup and tasks which
0607 aren't switching cgroup can be skipped easily using the
0608 cgroup_taskset_for_each() iterator. Note that this isn't called on a
0609 fork. If this method returns 0 (success) then this should remain valid
0610 while the caller holds cgroup_mutex and it is ensured that either
0611 attach() or cancel_attach() will be called in future.
0612 
0613 ``void css_reset(struct cgroup_subsys_state *css)``
0614 (cgroup_mutex held by caller)
0615 
0616 An optional operation which should restore @css's configuration to the
0617 initial state.  This is currently only used on the unified hierarchy
0618 when a subsystem is disabled on a cgroup through
0619 "cgroup.subtree_control" but should remain enabled because other
0620 subsystems depend on it.  cgroup core makes such a css invisible by
0621 removing the associated interface files and invokes this callback so
0622 that the hidden subsystem can return to the initial neutral state.
0623 This prevents unexpected resource control from a hidden css and
0624 ensures that the configuration is in the initial state when it is made
0625 visible again later.
0626 
0627 ``void cancel_attach(struct cgroup *cgrp, struct cgroup_taskset *tset)``
0628 (cgroup_mutex held by caller)
0629 
0630 Called when a task attach operation has failed after can_attach() has succeeded.
0631 A subsystem whose can_attach() has some side-effects should provide this
0632 function, so that the subsystem can implement a rollback. If not, not necessary.
0633 This will be called only about subsystems whose can_attach() operation have
0634 succeeded. The parameters are identical to can_attach().
0635 
0636 ``void attach(struct cgroup *cgrp, struct cgroup_taskset *tset)``
0637 (cgroup_mutex held by caller)
0638 
0639 Called after the task has been attached to the cgroup, to allow any
0640 post-attachment activity that requires memory allocations or blocking.
0641 The parameters are identical to can_attach().
0642 
0643 ``void fork(struct task_struct *task)``
0644 
0645 Called when a task is forked into a cgroup.
0646 
0647 ``void exit(struct task_struct *task)``
0648 
0649 Called during task exit.
0650 
0651 ``void free(struct task_struct *task)``
0652 
0653 Called when the task_struct is freed.
0654 
0655 ``void bind(struct cgroup *root)``
0656 (cgroup_mutex held by caller)
0657 
0658 Called when a cgroup subsystem is rebound to a different hierarchy
0659 and root cgroup. Currently this will only involve movement between
0660 the default hierarchy (which never has sub-cgroups) and a hierarchy
0661 that is being created/destroyed (and hence has no sub-cgroups).
0662 
0663 4. Extended attribute usage
0664 ===========================
0665 
0666 cgroup filesystem supports certain types of extended attributes in its
0667 directories and files.  The current supported types are:
0668 
0669         - Trusted (XATTR_TRUSTED)
0670         - Security (XATTR_SECURITY)
0671 
0672 Both require CAP_SYS_ADMIN capability to set.
0673 
0674 Like in tmpfs, the extended attributes in cgroup filesystem are stored
0675 using kernel memory and it's advised to keep the usage at minimum.  This
0676 is the reason why user defined extended attributes are not supported, since
0677 any user can do it and there's no limit in the value size.
0678 
0679 The current known users for this feature are SELinux to limit cgroup usage
0680 in containers and systemd for assorted meta data like main PID in a cgroup
0681 (systemd creates a cgroup per service).
0682 
0683 5. Questions
0684 ============
0685 
0686 ::
0687 
0688   Q: what's up with this '/bin/echo' ?
0689   A: bash's builtin 'echo' command does not check calls to write() against
0690      errors. If you use it in the cgroup file system, you won't be
0691      able to tell whether a command succeeded or failed.
0692 
0693   Q: When I attach processes, only the first of the line gets really attached !
0694   A: We can only return one error code per call to write(). So you should also
0695      put only ONE PID.