diff -uNrp linux-2.6.13/Documentation/adeos.txt linux-2.6.13-adeos/Documentation/adeos.txt --- linux-2.6.13/Documentation/adeos.txt 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/Documentation/adeos.txt 2004-11-24 12:33:17.000000000 +0100 @@ -0,0 +1,176 @@ + +The Adeos nanokernel is based on research and publications made in the +early '90s on the subject of nanokernels. Our basic method was to +reverse the approach described in most of the papers on the subject. +Instead of first building the nanokernel and then building the client +OSes, we started from a live and known-to-be-functional OS, Linux, and +inserted a nanokernel beneath it. Starting from Adeos, other client +OSes can now be put side-by-side with the Linux kernel. + +To this end, Adeos enables multiple domains to exist simultaneously on +the same hardware. None of these domains see each other, but all of +them see Adeos. A domain is most probably a complete OS, but there is +no assumption being made regarding the sophistication of what's in +a domain. + +To share the hardware among the different OSes, Adeos implements an +interrupt pipeline (ipipe). Every OS domain has an entry in the ipipe. +Each interrupt that comes in the ipipe is passed on to every domain +in the ipipe. Instead of disabling/enabling interrupts, each domain +in the pipeline only needs to stall/unstall his pipeline stage. If +an ipipe stage is stalled, then the interrupts do not progress in the +ipipe until that stage has been unstalled. Each stage of the ipipe +can, of course, decide to do a number of things with an interrupt. +Among other things, it can decide that it's the last recipient of the +interrupt. In that case, the ipipe does not propagate the interrupt +to the rest of the domains in the ipipe. + +Regardless of the operations being done in the ipipe, the Adeos code +does __not__ play with the interrupt masks. The only case where the +hardware masks are altered is during the addition/removal of a domain +from the ipipe. This also means that no OS is allowed to use the real +hardware cli/sti. But this is OK, since the stall/unstall calls +achieve the same functionality. + +Our approach is based on the following papers (links to these +papers are provided at the bottom of this message): +[1] D. Probert, J. Bruno, and M. Karzaorman. "Space: a new approach to +operating system abstraction." In: International Workshop on Object +Orientation in Operating Systems, pages 133-137, October 1991. +[2] D. Probert, J. Bruno. "Building fundamentally extensible application- +specific operating systems in Space", March 1995. +[3] D. Cheriton, K. Duda. "A caching model of operating system kernel +functionality". In: Proc. Symp. on Operating Systems Design and +Implementation, pages 179-194, Monterey CA (USA), 1994. +[4] D. Engler, M. Kaashoek, and J. O'Toole Jr. "Exokernel: an operating +system architecture for application-specific resource management", +December 1995. + +If you don't want to go fetch the complete papers, here's a summary. +The first 2 discuss the Space nanokernel, the 3rd discussed the cache +nanokernel, and the last discusses exokernel. + +The complete Adeos approach has been thoroughly documented in a whitepaper +published more than a year ago entitled "Adaptive Domain Environment +for Operating Systems" and available here: http://www.opersys.com/adeos +The current implementation is slightly different. Mainly, we do not +implement the functionality to move Linux out of ring 0. Although of +interest, this approach is not very portable. + +Instead, our patch taps right into Linux's main source of control +over the hardware, the interrupt dispatching code, and inserts an +interrupt pipeline which can then serve all the nanokernel's clients, +including Linux. + +This is not a novelty in itself. Other OSes have been modified in such +a way for a wide range of purposes. One of the most interesting +examples is described by Stodolsky, Chen, and Bershad in a paper +entitled "Fast Interrupt Priority Management in Operating System +Kernels" published in 1993 as part of the Usenix Microkernels and +Other Kernel Architectures Symposium. In that case, cli/sti were +replaced by virtual cli/sti which did not modify the real interrupt +mask in any way. Instead, interrupts were defered and delivered to +the OS upon a call to the virtualized sti. + +Mainly, this resulted in increased performance for the OS. Although +we haven't done any measurements on Linux's interrupt handling +performance with Adeos, our nanokernel includes by definition the +code implementing the technique described in the abovementioned +Stodolsky paper, which we use to redirect the hardware interrupt flow +to the pipeline. + +i386 and armnommu are currently supported. Most of the +architecture-dependent code is easily portable to other architectures. + +Aside of adding the Adeos module (driver/adeos), we also modified some +files to tap into Linux interrupt and system event dispatching (all +the modifications are encapsulated in #ifdef CONFIG_ADEOS_*/#endif). + +We modified the idle task so it gives control back to Adeos in order for +the ipipe to continue propagation. + +We modified init/main.c to initialize Adeos very early in the startup. + +Of course, we also added the appropriate makefile modifications and +config options so that you can choose to enable/disable Adeos as +part of the kernel build configuration. + +Adeos' public API is fully documented here: +http://www.freesoftware.fsf.org/adeos/doc/api/index.html. + +In Linux's case, adeos_register_domain() is called very early during +system startup. + +To add your domain to the ipipe, you need to: +1) Register your domain with Adeos using adeos_register_domain() +2) Call adeos_virtualize_irq() for all the IRQs you wish to be +notified about in the ipipe. + +That's it. Provided you gave Adeos appropriate handlers in step +#2, your interrupts will be delivered via the ipipe. + +During runtime, you may change your position in the ipipe using +adeos_renice_domain(). You may also stall/unstall the pipeline +and change the ipipe's handling of the interrupts according to your +needs. + +Adeos supports SMP, and APIC support on UP. + +Here are some of the possible uses for Adeos (this list is far +from complete): +1) Much like User-Mode Linux, it should now be possible to have 2 +Linux kernels living side-by-side on the same hardware. In contrast +to UML, this would not be 2 kernels one ontop of the other, but +really side-by-side. Since Linux can be told at boot time to use +only one portion of the available RAM, on a 128MB machine this +would mean that the first could be made to use the 0-64MB space and +the second would use the 64-128MB space. We realize that many +modifications are required. Among other things, one of the 2 kernels +will not need to conduct hardware initialization. Nevertheless, this +possibility should be studied closer. + +2) It follows from #1 that adding other kernels beside Linux should +be feasible. BSD is a prime candidate, but it would also be nice to +see what virtualizers such as VMWare and Plex86 could do with Adeos. +Proprietary operating systems could potentially also be accomodated. + +3) All the previous work that has been done on nanokernels should now +be easily ported to Linux. Mainly, we would be very interested to +hear about extensions to Adeos. Primarily, we have no mechanisms +currently enabling multiple domains to share information. The papers +mentioned earlier provide such mechanisms, but we'd like to see +actual practical examples. + +4) Kernel debuggers' main problem (tapping into the kernel's +interrupts) is solved and it should then be possible to provide +patchless kernel debuggers. They would then become loadable kernel +modules. + +5) Drivers who require absolute priority and dislike other kernel +portions who use cli/sti can now create a domain of their own +and place themselves before Linux in the ipipe. This provides a +mechanism for the implementation of systems that can provide guaranteed +realtime response. + +Philippe Gerum +Karim Yaghmour + +---------------------------------------------------------------------- +Links to papers: +1- +http://citeseer.nj.nec.com/probert91space.html +ftp://ftp.cs.ucsb.edu/pub/papers/space/iwooos91.ps.gz (not working) +http://www4.informatik.uni-erlangen.de/~tsthiel/Papers/Space-iwooos91.ps.gz + +2- +http://www.cs.ucsb.edu/research/trcs/abstracts/1995-06.shtml +http://www4.informatik.uni-erlangen.de/~tsthiel/Papers/Space-trcs95-06.ps.gz + +3- +http://citeseer.nj.nec.com/kenneth94caching.html +http://guir.cs.berkeley.edu/projects/osprelims/papers/cachmodel-OSkernel.ps.gz + +4- +http://citeseer.nj.nec.com/engler95exokernel.html +ftp://ftp.cag.lcs.mit.edu/multiscale/exokernel.ps.Z +---------------------------------------------------------------------- diff -uNrp linux-2.6.13/Makefile linux-2.6.13-adeos/Makefile --- linux-2.6.13/Makefile 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/Makefile 2005-09-04 12:38:02.000000000 +0200 @@ -564,6 +564,8 @@ export MODLIB ifeq ($(KBUILD_EXTMOD),) core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ +core-$(CONFIG_ADEOS) += adeos/ + vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \ $(net-y) $(net-m) $(libs-y) $(libs-m))) diff -uNrp linux-2.6.13/adeos/Kconfig linux-2.6.13-adeos/adeos/Kconfig --- linux-2.6.13/adeos/Kconfig 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/adeos/Kconfig 2005-09-07 15:02:32.000000000 +0200 @@ -0,0 +1,36 @@ +menu "Adeos support" + +config ADEOS + tristate "Adeos support" + default y + ---help--- + Activate this option if you want the Adeos nanokernel to be + compiled in. + +config ADEOS_CORE + def_bool ADEOS + +config ADEOS_THREADS + bool "Threaded domains" + depends on ADEOS + default n + ---help--- + This option causes the domains to run as lightweight + threads, which is useful for having seperate stacks + for them. If disabled, interrupts/events are directly + processed on behalf of the preempted context. Say N if + unsure. + +config ADEOS_NOTHREADS + def_bool !ADEOS_THREADS + +config ADEOS_PROFILING + bool "Pipeline profiling" + depends on ADEOS + default n + ---help--- + This option activates the profiling code which collects the + timestamps needed to measure the propagation time of + interrupts through the pipeline. Say N if unsure. + +endmenu diff -uNrp linux-2.6.13/adeos/Makefile linux-2.6.13-adeos/adeos/Makefile --- linux-2.6.13/adeos/Makefile 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/adeos/Makefile 2005-05-27 19:17:37.000000000 +0200 @@ -0,0 +1,13 @@ +# +# Makefile for the Adeos layer. +# + +obj-$(CONFIG_ADEOS) += adeos.o + +adeos-objs := generic.o + +adeos-$(CONFIG_X86) += x86.o + +adeos-$(CONFIG_IA64) += ia64.o + +adeos-$(CONFIG_PPC) += ppc.o diff -uNrp linux-2.6.13/adeos/generic.c linux-2.6.13-adeos/adeos/generic.c --- linux-2.6.13/adeos/generic.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/adeos/generic.c 2005-07-24 19:34:54.000000000 +0200 @@ -0,0 +1,640 @@ +/* + * linux/adeos/generic.c + * + * Copyright (C) 2002 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent ADEOS services. + */ + +#include +#include +#include +#include +#include +#include + +MODULE_DESCRIPTION("Adeos nanokernel"); +MODULE_AUTHOR("Philippe Gerum"); +MODULE_LICENSE("GPL"); + +/* adeos_register_domain() -- Add a new domain to the system. All + client domains must call this routine to register themselves to + ADEOS before using its services. */ + +int adeos_register_domain (adomain_t *adp, adattr_t *attr) + +{ + struct list_head *pos; + unsigned long flags; + int n; + + if (adp_current != adp_root) + { + printk(KERN_WARNING "Adeos: Only the root domain may register a new domain.\n"); + return -EPERM; + } + + flags = adeos_critical_enter(NULL); + + list_for_each(pos,&__adeos_pipeline) { + adomain_t *_adp = list_entry(pos,adomain_t,p_link); + if (_adp->domid == attr->domid) + break; + } + + adeos_critical_exit(flags); + + if (pos != &__adeos_pipeline) + /* A domain with the given id already exists -- fail. */ + return -EBUSY; + + for (n = 0; n < ADEOS_NR_CPUS; n++) + { + /* Each domain starts in sleeping state on every CPU. */ + adp->cpudata[n].status = (1 << IPIPE_SLEEP_FLAG); +#ifdef CONFIG_ADEOS_THREADS + adp->estackbase[n] = 0; +#endif /* CONFIG_ADEOS_THREADS */ + } + + adp->name = attr->name; + adp->priority = attr->priority; + adp->domid = attr->domid; + adp->dswitch = attr->dswitch; + adp->flags = 0; + adp->ptd_setfun = attr->ptdset; + adp->ptd_getfun = attr->ptdget; + adp->ptd_keymap = 0; + adp->ptd_keycount = 0; + adp->ptd_keymax = attr->nptdkeys; + + for (n = 0; n < ADEOS_NR_EVENTS; n++) + /* Event handlers must be cleared before the i-pipe stage is + inserted since an exception may occur on behalf of the new + emerging domain. */ + adp->events[n].handler = NULL; + + if (attr->entry != NULL) + __adeos_init_domain(adp,attr); + + /* Insert the domain in the interrupt pipeline last, so it won't + be resumed for processing interrupts until it has a valid stack + context. */ + + __adeos_init_stage(adp); + + INIT_LIST_HEAD(&adp->p_link); + + flags = adeos_critical_enter(NULL); + + list_for_each(pos,&__adeos_pipeline) { + adomain_t *_adp = list_entry(pos,adomain_t,p_link); + if (adp->priority > _adp->priority) + break; + } + + list_add_tail(&adp->p_link,pos); + + adeos_critical_exit(flags); + + printk(KERN_WARNING "Adeos: Domain %s registered.\n",adp->name); + + /* Finally, allow the new domain to perform its initialization + chores. */ + + if (attr->entry != NULL) + { + adeos_declare_cpuid; + + adeos_lock_cpu(flags); + +#ifdef CONFIG_ADEOS_THREADS + __adeos_switch_to(adp_root,adp,cpuid); +#else /* !CONFIG_ADEOS_THREADS */ + adp_cpu_current[cpuid] = adp; + attr->entry(1); + adp_cpu_current[cpuid] = adp_root; +#endif /* CONFIG_ADEOS_THREADS */ + + adeos_load_cpuid(); /* Processor might have changed. */ + + if (adp_root->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status)) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + + adeos_unlock_cpu(flags); + } + + return 0; +} + +/* adeos_unregister_domain() -- Remove a domain from the system. All + client domains must call this routine to unregister themselves from + the ADEOS layer. */ + +int adeos_unregister_domain (adomain_t *adp) + +{ + unsigned long flags; + unsigned event; + + if (adp_current != adp_root) + { + printk(KERN_WARNING "Adeos: Only the root domain may unregister a domain.\n"); + return -EPERM; + } + + if (adp == adp_root) + { + printk(KERN_WARNING "Adeos: Cannot unregister the root domain.\n"); + return -EPERM; + } + + for (event = 0; event < ADEOS_NR_EVENTS; event++) + /* Need this to update the monitor count. */ + adeos_catch_event_from(adp,event,NULL); + +#ifdef CONFIG_SMP + { + int nr_cpus = num_online_cpus(), _cpuid; + unsigned irq; + + /* In the SMP case, wait for the logged events to drain on other + processors before eventually removing the domain from the + pipeline. */ + + adeos_unstall_pipeline_from(adp); + + flags = adeos_critical_enter(NULL); + + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) + { + clear_bit(IPIPE_HANDLE_FLAG,&adp->irqs[irq].control); + clear_bit(IPIPE_STICKY_FLAG,&adp->irqs[irq].control); + set_bit(IPIPE_PASS_FLAG,&adp->irqs[irq].control); + } + + adeos_critical_exit(flags); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + { + for (irq = 0; irq < IPIPE_NR_IRQS; irq++) + while (adp->cpudata[_cpuid].irq_hits[irq] > 0) + cpu_relax(); + + while (test_bit(IPIPE_XPEND_FLAG,&adp->cpudata[_cpuid].status)) + cpu_relax(); + + while (!test_bit(IPIPE_SLEEP_FLAG,&adp->cpudata[_cpuid].status)) + cpu_relax(); + } + } +#endif /* CONFIG_SMP */ + + /* Simply remove the domain from the pipeline and we are almost + done. */ + + flags = adeos_critical_enter(NULL); + list_del_init(&adp->p_link); + adeos_critical_exit(flags); + + __adeos_cleanup_domain(adp); + + printk(KERN_WARNING "Adeos: Domain %s unregistered.\n",adp->name); + + return 0; +} + +/* adeos_propagate_irq() -- Force a given IRQ propagation on behalf of + a running interrupt handler to the next domain down the pipeline. + Returns non-zero if a domain has received the interrupt + notification, zero otherwise. + This call is useful for handling shared interrupts among domains. + e.g. pipeline = [domain-A]---[domain-B]... + Both domains share IRQ #X. + - domain-A handles IRQ #X but does not pass it down (i.e. Terminate + or Dynamic interrupt control mode) + - domain-B handles IRQ #X (i.e. Terminate or Accept interrupt + control modes). + When IRQ #X is raised, domain-A's handler determines whether it + should process the interrupt by identifying its source. If not, + adeos_propagate_irq() is called so that the next domain down the + pipeline which handles IRQ #X is given a chance to process it. This + process can be repeated until the end of the pipeline is + reached. */ + +/* adeos_schedule_irq() -- Almost the same as adeos_propagate_irq(), + but attempts to pend the interrupt for the current domain first. */ + +int fastcall __adeos_schedule_irq (unsigned irq, struct list_head *head) + +{ + struct list_head *ln; + unsigned long flags; + adeos_declare_cpuid; + + if (irq >= IPIPE_NR_IRQS || + (adeos_virtual_irq_p(irq) && !test_bit(irq - IPIPE_VIRQ_BASE,&__adeos_virtual_irq_map))) + return -EINVAL; + + adeos_lock_cpu(flags); + + ln = head; + + while (ln != &__adeos_pipeline) + { + adomain_t *adp = list_entry(ln,adomain_t,p_link); + + if (test_bit(IPIPE_HANDLE_FLAG,&adp->irqs[irq].control)) + { + adp->cpudata[cpuid].irq_hits[irq]++; + __adeos_set_irq_bit(adp,cpuid,irq); + adeos_unlock_cpu(flags); + return 1; + } + + ln = adp->p_link.next; + } + + adeos_unlock_cpu(flags); + + return 0; +} + +/* adeos_free_irq() -- Return a previously allocated virtual/soft + pipelined interrupt to the pool of allocatable interrupts. */ + +int adeos_free_irq (unsigned irq) + +{ + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + clear_bit(irq - IPIPE_VIRQ_BASE,&__adeos_virtual_irq_map); + + return 0; +} + +cpumask_t adeos_set_irq_affinity (unsigned irq, cpumask_t cpumask) + +{ +#ifdef CONFIG_SMP + if (irq >= IPIPE_NR_XIRQS) + /* Allow changing affinity of external IRQs only. */ + return CPU_MASK_NONE; + + if (num_online_cpus() > 1) + /* Allow changing affinity of external IRQs only. */ + return __adeos_set_irq_affinity(irq,cpumask); +#endif /* CONFIG_SMP */ + + return CPU_MASK_NONE; +} + +/* adeos_catch_event_from() -- Interpose an event handler starting + from a given domain. */ + +adevhand_t adeos_catch_event_from (adomain_t *adp, unsigned event, adevhand_t handler) + +{ + adevhand_t oldhandler; + + if (event >= ADEOS_NR_EVENTS) + return NULL; + + if ((oldhandler = (adevhand_t)xchg(&adp->events[event].handler,handler)) == NULL) + { + if (handler) + __adeos_event_monitors[event]++; + } + else if (!handler) + __adeos_event_monitors[event]--; + + return oldhandler; +} + +void adeos_init_attr (adattr_t *attr) + +{ + attr->name = "Anonymous"; + attr->domid = 1; + attr->entry = NULL; + attr->estacksz = 0; /* Let ADEOS choose a reasonable stack size */ + attr->priority = ADEOS_ROOT_PRI; + attr->dswitch = NULL; + attr->nptdkeys = 0; + attr->ptdset = NULL; + attr->ptdget = NULL; +} + +int adeos_alloc_ptdkey (void) + +{ + unsigned long flags; + int key = -1; + + spin_lock_irqsave_hw(&__adeos_pipelock,flags); + + if (adp_current->ptd_keycount < adp_current->ptd_keymax) + { + key = ffz(adp_current->ptd_keymap); + set_bit(key,&adp_current->ptd_keymap); + adp_current->ptd_keycount++; + } + + spin_unlock_irqrestore_hw(&__adeos_pipelock,flags); + + return key; +} + +int adeos_free_ptdkey (int key) + +{ + unsigned long flags; + + if (key < 0 || key >= adp_current->ptd_keymax) + return -EINVAL; + + spin_lock_irqsave_hw(&__adeos_pipelock,flags); + + if (test_and_clear_bit(key,&adp_current->ptd_keymap)) + adp_current->ptd_keycount--; + + spin_unlock_irqrestore_hw(&__adeos_pipelock,flags); + + return 0; +} + +int adeos_set_ptd (int key, void *value) + +{ + if (key < 0 || key >= adp_current->ptd_keymax) + return -EINVAL; + + if (!adp_current->ptd_setfun) + { + printk(KERN_WARNING "Adeos: No ptdset hook for %s\n",adp_current->name); + return -EINVAL; + } + + adp_current->ptd_setfun(key,value); + + return 0; +} + +void *adeos_get_ptd (int key) + +{ + if (key < 0 || key >= adp_current->ptd_keymax) + return NULL; + + if (!adp_current->ptd_getfun) + { + printk(KERN_WARNING "Adeos: No ptdget hook for %s\n",adp_current->name); + return NULL; + } + + return adp_current->ptd_getfun(key); +} + +int adeos_init_mutex (admutex_t *mutex) + +{ + admutex_t initm = ADEOS_MUTEX_UNLOCKED; + *mutex = initm; + return 0; +} + +#ifdef CONFIG_ADEOS_THREADS + +int adeos_destroy_mutex (admutex_t *mutex) + +{ + if (!adeos_spin_trylock(&mutex->lock) && + adp_current != adp_root && + mutex->owner != adp_current) + return -EBUSY; + + return 0; +} + +static inline void __adeos_sleepon_mutex (admutex_t *mutex, adomain_t *sleeper, int cpuid) + +{ + adomain_t *owner = mutex->owner; + + /* Make the current domain (== sleeper) wait for the mutex to be + released. Adeos' pipelined scheme guarantees that the new + sleeper _is_ higher priority than any aslept domain since we + have stalled each sleeper's stage. Must be called with local hw + interrupts off. */ + + sleeper->m_link = mutex->sleepq; + mutex->sleepq = sleeper; + __adeos_switch_to(adp_cpu_current[cpuid],owner,cpuid); + mutex->owner = sleeper; + adeos_spin_unlock(&mutex->lock); +} + +unsigned long fastcall adeos_lock_mutex (admutex_t *mutex) + +{ + unsigned long flags, hwflags; + adeos_declare_cpuid; + adomain_t *adp; + + if (!adp_pipelined) + { + adeos_hw_local_irq_save(hwflags); + flags = !adeos_hw_test_iflag(hwflags); + adeos_spin_lock(&mutex->lock); + return flags; + } + + adeos_lock_cpu(hwflags); + + adp = adp_cpu_current[cpuid]; + + flags = __test_and_set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + /* Two cases to handle here on SMP systems, only one for UP: 1) in + case of a conflicting access from a higher priority domain + running on the same cpu, make this domain sleep on the mutex, + and resume the current owner so it can release the lock asap. + 2) in case of a conflicting access from any domain on a + different cpu than the current owner's, simply enter a spinning + loop. Note that testing mutex->owncpu is safe since it is only + changed by the current owner, and set to -1 when the mutex is + unlocked. */ + +#ifdef CONFIG_SMP + while (!adeos_spin_trylock(&mutex->lock)) + { + if (mutex->owncpu == cpuid) + { + __adeos_sleepon_mutex(mutex,adp,cpuid); + adeos_load_cpuid(); + } + } + + mutex->owncpu = cpuid; +#else /* !CONFIG_SMP */ + while (mutex->owner != NULL && mutex->owner != adp) + __adeos_sleepon_mutex(mutex,adp,cpuid); +#endif /* CONFIG_SMP */ + + mutex->owner = adp; + + adeos_unlock_cpu(hwflags); + + return flags; +} + +void fastcall adeos_unlock_mutex (admutex_t *mutex, unsigned long flags) + +{ + unsigned long hwflags; + adeos_declare_cpuid; + adomain_t *adp; + + if (!adp_pipelined) + { + adeos_spin_unlock(&mutex->lock); + + if (flags) + adeos_hw_cli(); + else + adeos_hw_sti(); + + return; + } + +#ifdef CONFIG_SMP + mutex->owncpu = -1; +#endif /* CONFIG_SMP */ + + if (!flags) + adeos_hw_sti(); /* Absolutely needed. */ + + adeos_lock_cpu(hwflags); + + if (mutex->sleepq != NULL) + { + adomain_t *sleeper = mutex->sleepq; + /* Wake up the highest priority sleeper. */ + mutex->sleepq = sleeper->m_link; + __adeos_switch_to(adp_cpu_current[cpuid],sleeper,cpuid); + adeos_load_cpuid(); + } + else + { + mutex->owner = NULL; + adeos_spin_unlock(&mutex->lock); + } + + adp = adp_cpu_current[cpuid]; + + if (flags) + __set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + else + { + __clear_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + if (adp->cpudata[cpuid].irq_pending_hi != 0) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + } + + adeos_unlock_cpu(hwflags); +} + +#else /* !CONFIG_ADEOS_THREADS */ + +int adeos_destroy_mutex (admutex_t *mutex) + +{ + if (!adeos_spin_trylock(&mutex->lock) && + adp_current != adp_root) + return -EBUSY; + + return 0; +} + +unsigned long fastcall adeos_lock_mutex (admutex_t *mutex) + +{ + unsigned long flags; /* FIXME: won't work on SPARC */ + spin_lock_irqsave_hw(&mutex->lock,flags); + return flags; +} + +void fastcall adeos_unlock_mutex (admutex_t *mutex, unsigned long flags) + +{ + spin_unlock_irqrestore_hw(&mutex->lock,flags); +} + +#endif /* CONFIG_ADEOS_THREADS */ + +void __adeos_takeover (void) + +{ + __adeos_enable_pipeline(); + printk(KERN_WARNING "Adeos: Pipelining started.\n"); +} + +#ifdef MODULE + +static int __init adeos_init_module (void) + +{ + __adeos_takeover(); + return 0; +} + +static void __exit adeos_exit_module (void) + +{ + __adeos_disable_pipeline(); + printk(KERN_WARNING "Adeos: Pipelining stopped.\n"); +} + +module_init(adeos_init_module); +module_exit(adeos_exit_module); + +#endif /* MODULE */ + +EXPORT_SYMBOL(adeos_register_domain); +EXPORT_SYMBOL(adeos_unregister_domain); +EXPORT_SYMBOL(adeos_virtualize_irq_from); +EXPORT_SYMBOL(adeos_control_irq); +EXPORT_SYMBOL(__adeos_schedule_irq); +EXPORT_SYMBOL(adeos_free_irq); +EXPORT_SYMBOL(adeos_send_ipi); +EXPORT_SYMBOL(adeos_catch_event_from); +EXPORT_SYMBOL(adeos_init_attr); +EXPORT_SYMBOL(adeos_get_sysinfo); +EXPORT_SYMBOL(adeos_tune_timer); +EXPORT_SYMBOL(adeos_alloc_ptdkey); +EXPORT_SYMBOL(adeos_free_ptdkey); +EXPORT_SYMBOL(adeos_set_ptd); +EXPORT_SYMBOL(adeos_get_ptd); +EXPORT_SYMBOL(adeos_set_irq_affinity); +EXPORT_SYMBOL(adeos_init_mutex); +EXPORT_SYMBOL(adeos_destroy_mutex); +EXPORT_SYMBOL(adeos_lock_mutex); +EXPORT_SYMBOL(adeos_unlock_mutex); diff -uNrp linux-2.6.13/adeos/x86.c linux-2.6.13-adeos/adeos/x86.c --- linux-2.6.13/adeos/x86.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/adeos/x86.c 2005-06-28 16:53:35.000000000 +0200 @@ -0,0 +1,778 @@ +/* + * linux/adeos/x86.c + * + * Copyright (C) 2002 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent ADEOS support for x86. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef CONFIG_X86_LOCAL_APIC +#include +#include +#include +#ifdef CONFIG_X86_IO_APIC +#include +#endif /* CONFIG_X86_IO_APIC */ +#include +#include +#endif /* CONFIG_X86_LOCAL_APIC */ + +extern struct desc_struct idt_table[]; + +extern void (*__adeos_irq_trampolines[])(void); /* in entry.S */ + +static void (*__adeos_std_vector_table[256])(void); + +/* Lifted from arch/i386/kernel/traps.c */ +#define __adeos_set_gate(gate_addr,type,dpl,addr) \ +do { \ + int __d0, __d1; \ + __asm__ __volatile__ ("movw %%dx,%%ax\n\t" \ + "movw %4,%%dx\n\t" \ + "movl %%eax,%0\n\t" \ + "movl %%edx,%1" \ + :"=m" (*((long *) (gate_addr))), \ + "=m" (*(1+(long *) (gate_addr))), "=&a" (__d0), "=&d" (__d1) \ + :"i" ((short) (0x8000+(dpl<<13)+(type<<8))), \ + "3" ((char *) (addr)),"2" (__KERNEL_CS << 16)); \ +} while (0) + +#define __adeos_get_gate_addr(v) \ + ((void *)((idt_table[v].b & 0xffff0000)|(idt_table[v].a & 0xffff))) + +#define __adeos_set_irq_gate(vector,addr) \ +__adeos_set_gate(idt_table+vector,14,0,addr) + +#define __adeos_set_trap_gate(vector,addr) \ +__adeos_set_gate(idt_table+vector,15,0,addr) + +#define __adeos_set_sys_gate(vector,addr) \ +__adeos_set_gate(idt_table+vector,15,3,addr) + +#define BUILD_TRAP_PROTO(trapnr) __adeos_trap_##trapnr(void) + +#define BUILD_TRAP_ERRCODE(trapnr) \ +asmlinkage void BUILD_TRAP_PROTO(trapnr); \ +__asm__ ( \ + "\n" __ALIGN_STR"\n\t" \ + SYMBOL_NAME_STR(__adeos_trap_) #trapnr ":\n\t" \ + "cld\n\t" \ + "pushl %es\n\t" \ + "pushl %ds\n\t" \ + "pushl %eax\n\t" \ + "pushl %ebp\n\t" \ + "pushl %edi\n\t" \ + "pushl %esi\n\t" \ + "pushl %edx\n\t" \ + "pushl %ecx\n\t" \ + "pushl %ebx\n\t" \ + "movl $" STR(__USER_DS) ",%edx\n\t" \ + "mov %dx,%ds\n\t" \ + "mov %dx,%es\n\t" \ + "movl ("SYMBOL_NAME_STR(__adeos_event_monitors + 4 * trapnr)"),%eax\n\t" \ + "testl %eax,%eax\n\t" \ + "jz 1f\n\t" \ + "movl %esp,%eax\n\t" \ + "pushl %eax\n\t" \ + "pushl $"#trapnr"\n\t" \ + "call " SYMBOL_NAME_STR(__adeos_handle_event) "\n\t" \ + "addl $8,%esp\n\t" \ + "testl %eax,%eax\n\t" \ +"1: popl %ebx\n\t" \ + "popl %ecx\n\t" \ + "popl %edx\n\t" \ + "popl %esi\n\t" \ + "popl %edi\n\t" \ + "popl %ebp\n\t" \ + "jz 2f\n\t" \ + "popl %eax\n\t" \ + "popl %ds\n\t" \ + "popl %es\n\t" \ + "addl $4,%esp\n\t" \ + "iret\n" \ +"2: movl ("SYMBOL_NAME_STR(__adeos_std_vector_table + 4 * trapnr)"),%eax\n\t" \ + "mov 8(%esp),%es\n\t" \ + "movl %eax,8(%esp)\n\t" \ + "popl %eax\n\t" \ + "popl %ds\n\t" \ + "ret\n"); + +#define BUILD_TRAP_NOERRCODE(trapnr) \ +asmlinkage void BUILD_TRAP_PROTO(trapnr); \ +__asm__ ( \ + "\n" __ALIGN_STR"\n\t" \ + SYMBOL_NAME_STR(__adeos_trap_) #trapnr ":\n\t" \ + "cld\n\t" \ + "pushl $0\n\t" \ + "pushl %es\n\t" \ + "pushl %ds\n\t" \ + "pushl %eax\n\t" \ + "pushl %ebp\n\t" \ + "pushl %edi\n\t" \ + "pushl %esi\n\t" \ + "pushl %edx\n\t" \ + "pushl %ecx\n\t" \ + "pushl %ebx\n\t" \ + "movl $" STR(__USER_DS) ",%edx\n\t" \ + "mov %dx,%ds\n\t" \ + "mov %dx,%es\n\t" \ + "movl ("SYMBOL_NAME_STR(__adeos_event_monitors + 4 * trapnr)"),%eax\n\t" \ + "testl %eax,%eax\n\t" \ + "jz 1f\n\t" \ + "movl %esp,%eax\n\t" \ + "pushl %eax\n\t" \ + "pushl $"#trapnr"\n\t" \ + "call " SYMBOL_NAME_STR(__adeos_handle_event) "\n\t" \ + "addl $8,%esp\n\t" \ + "testl %eax,%eax\n\t" \ +"1: popl %ebx\n\t" \ + "popl %ecx\n\t" \ + "popl %edx\n\t" \ + "popl %esi\n\t" \ + "popl %edi\n\t" \ + "popl %ebp\n\t" \ + "jz 2f\n\t" \ + "popl %eax\n\t" \ + "popl %ds\n\t" \ + "popl %es\n\t" \ + "addl $4,%esp\n\t" \ + "iret\n" \ +"2: movl ("SYMBOL_NAME_STR(__adeos_std_vector_table + 4 * trapnr)"),%eax\n\t" \ + "mov %eax,12(%esp)\n\t" \ + "popl %eax\n\t" \ + "popl %ds\n\t" \ + "popl %es\n\t" \ + "ret\n"); + +BUILD_TRAP_NOERRCODE(0x0) BUILD_TRAP_NOERRCODE(0x1) BUILD_TRAP_NOERRCODE(0x2) +BUILD_TRAP_NOERRCODE(0x3) BUILD_TRAP_NOERRCODE(0x4) BUILD_TRAP_NOERRCODE(0x5) +BUILD_TRAP_NOERRCODE(0x6) BUILD_TRAP_NOERRCODE(0x7) BUILD_TRAP_ERRCODE(0x8) +BUILD_TRAP_NOERRCODE(0x9) BUILD_TRAP_ERRCODE(0xa) BUILD_TRAP_ERRCODE(0xb) +BUILD_TRAP_ERRCODE(0xc) BUILD_TRAP_ERRCODE(0xd) BUILD_TRAP_ERRCODE(0xe) +BUILD_TRAP_NOERRCODE(0xf) BUILD_TRAP_NOERRCODE(0x10) BUILD_TRAP_ERRCODE(0x11) +BUILD_TRAP_NOERRCODE(0x12) BUILD_TRAP_NOERRCODE(0x13) BUILD_TRAP_ERRCODE(0x14) +BUILD_TRAP_ERRCODE(0x15) BUILD_TRAP_ERRCODE(0x16) BUILD_TRAP_ERRCODE(0x17) +BUILD_TRAP_ERRCODE(0x18) BUILD_TRAP_ERRCODE(0x19) BUILD_TRAP_ERRCODE(0x1a) +BUILD_TRAP_ERRCODE(0x1b) BUILD_TRAP_ERRCODE(0x1c) BUILD_TRAP_ERRCODE(0x1d) +BUILD_TRAP_ERRCODE(0x1e) BUILD_TRAP_ERRCODE(0x1f) + +#define LIST_TRAP(trapnr) &__adeos_trap_ ## trapnr + +static void (*__adeos_trap_trampolines[])(void) = { + LIST_TRAP(0x0), LIST_TRAP(0x1), LIST_TRAP(0x2), LIST_TRAP(0x3), + LIST_TRAP(0x4), LIST_TRAP(0x5), LIST_TRAP(0x6), LIST_TRAP(0x7), + LIST_TRAP(0x8), LIST_TRAP(0x9), LIST_TRAP(0xa), LIST_TRAP(0xb), + LIST_TRAP(0xc), LIST_TRAP(0xd), LIST_TRAP(0xe), LIST_TRAP(0xf), + LIST_TRAP(0x10), LIST_TRAP(0x11), LIST_TRAP(0x12), LIST_TRAP(0x13), + LIST_TRAP(0x14), LIST_TRAP(0x15), LIST_TRAP(0x16), LIST_TRAP(0x17), + LIST_TRAP(0x18), LIST_TRAP(0x19), LIST_TRAP(0x1a), LIST_TRAP(0x1b), + LIST_TRAP(0x1c), LIST_TRAP(0x1d), LIST_TRAP(0x1e), LIST_TRAP(0x1f) +}; + +static int __adeos_ack_common_irq (unsigned irq) + +{ + irq_desc_t *desc = irq_desc + irq; + unsigned long flags; + adeos_declare_cpuid; + + adeos_load_cpuid(); /* hw interrupts are off. */ + flags = adeos_test_and_stall_pipeline(); + preempt_disable(); + desc->handler->ack(irq); + preempt_enable_no_resched(); + adeos_restore_pipeline_nosync(adp_cpu_current[cpuid],flags,cpuid); + + return 1; +} + +/* __adeos_enable_pipeline() -- Take over the interrupt control from + the root domain (i.e. Linux). After this routine has returned, all + interrupts go through the pipeline. */ + +void __adeos_enable_pipeline (void) + +{ + unsigned vector, irq, rirq; + unsigned long flags; + + /* Collect the original vector table. */ + + for (vector = 0; vector < 256; vector++) + __adeos_std_vector_table[vector] = __adeos_get_gate_addr(vector); + +#ifdef CONFIG_SMP + + /* This vector must be set up prior to call + adeos_critical_enter(). */ + + __adeos_set_irq_gate(ADEOS_CRITICAL_VECTOR, + __adeos_irq_trampolines[ADEOS_CRITICAL_IPI]); + +#endif /* CONFIG_SMP */ + + flags = adeos_critical_enter(NULL); + + /* First, grab the ISA and IO-APIC interrupts. */ + + for (irq = 0; irq < NR_IRQS && irq + FIRST_EXTERNAL_VECTOR < FIRST_SYSTEM_VECTOR; irq++) + { + rirq = irq; + +#ifdef CONFIG_X86_IO_APIC + if (IO_APIC_IRQ(irq)) + { + vector = IO_APIC_VECTOR(irq); + + if (vector == 0) + continue; + +#ifdef CONFIG_PCI_MSI + /* Account specifically for MSI routing. */ + if (!platform_legacy_irq(irq)) + rirq = vector; +#endif /* CONFIG_PCI_MSI */ + } + else +#endif /* CONFIG_X86_IO_APIC */ + { + vector = irq + FIRST_EXTERNAL_VECTOR; + + if (vector == SYSCALL_VECTOR) + continue; + } + + /* Fails for ADEOS_CRITICAL_IPI but that's ok. */ + + adeos_virtualize_irq(rirq, + (void (*)(unsigned))__adeos_std_vector_table[vector], + &__adeos_ack_common_irq, + IPIPE_CALLASM_MASK|IPIPE_HANDLE_MASK|IPIPE_PASS_MASK); + + __adeos_set_irq_gate(vector,__adeos_irq_trampolines[rirq]); + } + +#ifdef CONFIG_X86_LOCAL_APIC + + /* Map the APIC system vectors including the unused ones so that + client domains can virtualize the corresponding IRQs. */ + + for (vector = FIRST_SYSTEM_VECTOR; vector < CALL_FUNCTION_VECTOR; vector++) + { + adeos_virtualize_irq(vector - FIRST_EXTERNAL_VECTOR, + (void (*)(unsigned))__adeos_std_vector_table[vector], + &__adeos_ack_system_irq, + IPIPE_CALLASM_MASK|IPIPE_HANDLE_MASK|IPIPE_PASS_MASK); + + __adeos_set_irq_gate(vector, + __adeos_irq_trampolines[vector - FIRST_EXTERNAL_VECTOR]); + } + + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR0, + __adeos_irq_trampolines[ADEOS_SERVICE_IPI0]); + + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR1, + __adeos_irq_trampolines[ADEOS_SERVICE_IPI1]); + + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR2, + __adeos_irq_trampolines[ADEOS_SERVICE_IPI2]); + + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR3, + __adeos_irq_trampolines[ADEOS_SERVICE_IPI3]); + + __adeos_tick_irq = using_apic_timer ? LOCAL_TIMER_VECTOR - FIRST_EXTERNAL_VECTOR : 0; + +#else /* !CONFIG_X86_LOCAL_APIC */ + + __adeos_tick_irq = 0; + +#endif /* CONFIG_X86_LOCAL_APIC */ + +#ifdef CONFIG_SMP + + /* All interrupts must be pipelined, but the spurious one since we + don't even want to acknowledge it. */ + + for (vector = CALL_FUNCTION_VECTOR; vector < SPURIOUS_APIC_VECTOR; vector++) + { + adeos_virtualize_irq(vector - FIRST_EXTERNAL_VECTOR, + (void (*)(unsigned))__adeos_std_vector_table[vector], + &__adeos_ack_system_irq, + IPIPE_CALLASM_MASK|IPIPE_HANDLE_MASK|IPIPE_PASS_MASK); + + __adeos_set_irq_gate(vector, + __adeos_irq_trampolines[vector - FIRST_EXTERNAL_VECTOR]); + } + +#endif /* CONFIG_SMP */ + + /* Redirect traps and exceptions (except NMI). */ + + __adeos_set_trap_gate(0,__adeos_trap_trampolines[0]); + __adeos_set_trap_gate(1,__adeos_trap_trampolines[1]); + __adeos_set_sys_gate(3,__adeos_trap_trampolines[3]); + __adeos_set_sys_gate(4,__adeos_trap_trampolines[4]); + __adeos_set_sys_gate(5,__adeos_trap_trampolines[5]); + __adeos_set_trap_gate(6,__adeos_trap_trampolines[6]); + __adeos_set_trap_gate(7,__adeos_trap_trampolines[7]); + __adeos_set_trap_gate(8,__adeos_trap_trampolines[8]); + __adeos_set_trap_gate(9,__adeos_trap_trampolines[9]); + __adeos_set_trap_gate(10,__adeos_trap_trampolines[10]); + __adeos_set_trap_gate(11,__adeos_trap_trampolines[11]); + __adeos_set_trap_gate(12,__adeos_trap_trampolines[12]); + __adeos_set_trap_gate(13,__adeos_trap_trampolines[13]); + __adeos_set_irq_gate(14,__adeos_trap_trampolines[14]); + __adeos_set_trap_gate(15,__adeos_trap_trampolines[15]); + __adeos_set_trap_gate(16,__adeos_trap_trampolines[16]); + __adeos_set_trap_gate(17,__adeos_trap_trampolines[17]); + __adeos_set_trap_gate(18,__adeos_trap_trampolines[18]); + __adeos_set_trap_gate(19,__adeos_trap_trampolines[19]); + + adp_pipelined = 1; + + adeos_critical_exit(flags); +} + +/* __adeos_disable_pipeline() -- Disengage the pipeline. */ + +void __adeos_disable_pipeline (void) + +{ + unsigned vector, irq; + unsigned long flags; + + flags = adeos_critical_enter(NULL); + + /* Restore original IDT settings. */ + + for (irq = 0; irq < NR_IRQS && irq + FIRST_EXTERNAL_VECTOR < FIRST_SYSTEM_VECTOR; irq++) + { +#ifdef CONFIG_X86_IO_APIC + if (IO_APIC_IRQ(irq)) + { + vector = IO_APIC_VECTOR(irq); + + if (vector == 0) + continue; + } + else +#endif /* CONFIG_X86_IO_APIC */ + { + vector = irq + FIRST_EXTERNAL_VECTOR; + + if (vector == SYSCALL_VECTOR) + continue; + } + + __adeos_set_irq_gate(vector,__adeos_std_vector_table[vector]); + } + +#ifdef CONFIG_X86_LOCAL_APIC + + for (vector = FIRST_SYSTEM_VECTOR; vector < CALL_FUNCTION_VECTOR; vector++) + __adeos_set_irq_gate(vector,__adeos_std_vector_table[vector]); + + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR0,__adeos_std_vector_table[ADEOS_SERVICE_VECTOR0]); + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR1,__adeos_std_vector_table[ADEOS_SERVICE_VECTOR1]); + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR2,__adeos_std_vector_table[ADEOS_SERVICE_VECTOR2]); + __adeos_set_irq_gate(ADEOS_SERVICE_VECTOR3,__adeos_std_vector_table[ADEOS_SERVICE_VECTOR3]); + +#endif /* CONFIG_X86_LOCAL_APIC */ + +#ifdef CONFIG_SMP + + for (vector = CALL_FUNCTION_VECTOR; vector < SPURIOUS_APIC_VECTOR; vector++) + __adeos_set_irq_gate(vector,__adeos_std_vector_table[vector]); + + __adeos_set_irq_gate(ADEOS_CRITICAL_VECTOR,__adeos_std_vector_table[ADEOS_CRITICAL_VECTOR]); + +#endif /* CONFIG_SMP */ + + __adeos_set_trap_gate(0,__adeos_std_vector_table[0]); + __adeos_set_trap_gate(1,__adeos_std_vector_table[1]); + __adeos_set_sys_gate(3,__adeos_std_vector_table[3]); + __adeos_set_sys_gate(4,__adeos_std_vector_table[4]); + __adeos_set_sys_gate(5,__adeos_std_vector_table[5]); + __adeos_set_trap_gate(6,__adeos_std_vector_table[6]); + __adeos_set_trap_gate(7,__adeos_std_vector_table[7]); + __adeos_set_trap_gate(8,__adeos_std_vector_table[8]); + __adeos_set_trap_gate(9,__adeos_std_vector_table[9]); + __adeos_set_trap_gate(10,__adeos_std_vector_table[10]); + __adeos_set_trap_gate(11,__adeos_std_vector_table[11]); + __adeos_set_trap_gate(12,__adeos_std_vector_table[12]); + __adeos_set_trap_gate(13,__adeos_std_vector_table[13]); + __adeos_set_irq_gate(14,__adeos_std_vector_table[14]); + __adeos_set_trap_gate(15,__adeos_std_vector_table[15]); + __adeos_set_trap_gate(16,__adeos_std_vector_table[16]); + __adeos_set_trap_gate(17,__adeos_std_vector_table[17]); + __adeos_set_trap_gate(18,__adeos_std_vector_table[18]); + __adeos_set_trap_gate(19,__adeos_std_vector_table[19]); + + adp_pipelined = 0; + + adeos_critical_exit(flags); +} + +/* adeos_virtualize_irq_from() -- Attach a handler (and optionally a + hw acknowledge routine) to an interrupt for the given domain. */ + +int adeos_virtualize_irq_from (adomain_t *adp, + unsigned irq, + void (*handler)(unsigned irq), + int (*acknowledge)(unsigned irq), + unsigned modemask) +{ + unsigned long flags; + int err; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + if (adp->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + spin_lock_irqsave_hw(&__adeos_pipelock,flags); + + if (handler != NULL) + { + /* A bit of hack here: if we are re-virtualizing an IRQ just + to change the acknowledge routine by passing the special + ADEOS_SAME_HANDLER value, then allow to recycle the current + handler for the IRQ. This allows Linux device drivers + managing shared IRQ lines to call adeos_virtualize_irq() in + addition to request_irq() just for the purpose of + interposing their own shared acknowledge routine. */ + + if (handler == ADEOS_SAME_HANDLER) + { + handler = adp->irqs[irq].handler; + + if (handler == NULL) + { + err = -EINVAL; + goto unlock_and_exit; + } + } + else if ((modemask & IPIPE_EXCLUSIVE_MASK) != 0 && + adp->irqs[irq].handler != NULL) + { + err = -EBUSY; + goto unlock_and_exit; + } + + if ((modemask & (IPIPE_SHARED_MASK|IPIPE_PASS_MASK)) == IPIPE_SHARED_MASK) + { + err = -EINVAL; + goto unlock_and_exit; + } + + if ((modemask & IPIPE_STICKY_MASK) != 0) + modemask |= IPIPE_HANDLE_MASK; + } + else + modemask &= ~(IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SHARED_MASK); + + if (acknowledge == NULL) + { + if ((modemask & IPIPE_SHARED_MASK) == 0) + /* Acknowledge handler unspecified -- this is ok in + non-shared management mode, but we will force the use + of the Linux-defined handler instead. */ + acknowledge = adp_root->irqs[irq].acknowledge; + else + { + /* A valid acknowledge handler to be called in shared mode + is required when declaring a shared IRQ. */ + err = -EINVAL; + goto unlock_and_exit; + } + } + + adp->irqs[irq].handler = handler; + adp->irqs[irq].acknowledge = acknowledge; + adp->irqs[irq].control = modemask; + + if (irq < NR_IRQS && + handler != NULL && + !adeos_virtual_irq_p(irq) && + (modemask & IPIPE_ENABLE_MASK) != 0) + { + if (adp != adp_current) + { + /* IRQ enable/disable state is domain-sensitive, so we may + not change it for another domain. What is allowed + however is forcing some domain to handle an interrupt + source, by passing the proper 'adp' descriptor which + thus may be different from adp_current. */ + err = -EPERM; + goto unlock_and_exit; + } + + irq_desc[irq].handler->enable(irq); + } + + err = 0; + +unlock_and_exit: + + spin_unlock_irqrestore_hw(&__adeos_pipelock,flags); + + return err; +} + +/* adeos_control_irq() -- Change an interrupt mode. This affects the + way a given interrupt is handled by ADEOS for the current + domain. setmask is a bitmask telling whether: + - the interrupt should be passed to the domain (IPIPE_HANDLE_MASK), + and/or + - the interrupt should be passed down to the lower priority domain(s) + in the pipeline (IPIPE_PASS_MASK). + This leads to four possibilities: + - PASS only => Ignore the interrupt + - HANDLE only => Terminate the interrupt (process but don't pass down) + - PASS + HANDLE => Accept the interrupt (process and pass down) + - => Discard the interrupt + - DYNAMIC is currently an alias of HANDLE since it marks an interrupt + which is processed by the current domain but not implicitely passed + down to the pipeline, letting the domain's handler choose on a case- + by-case basis whether the interrupt propagation should be forced + using adeos_propagate_irq(). + clrmask clears the corresponding bits from the control field before + setmask is applied. +*/ + +int adeos_control_irq (unsigned irq, + unsigned clrmask, + unsigned setmask) +{ + unsigned long flags; + irq_desc_t *desc; + + if (irq >= IPIPE_NR_IRQS) + return -EINVAL; + + if (adp_current->irqs[irq].control & IPIPE_SYSTEM_MASK) + return -EPERM; + + if (((setmask|clrmask) & IPIPE_SHARED_MASK) != 0) + return -EINVAL; + + desc = irq_desc + irq; + + if (adp_current->irqs[irq].handler == NULL) + setmask &= ~(IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK); + + if ((setmask & IPIPE_STICKY_MASK) != 0) + setmask |= IPIPE_HANDLE_MASK; + + if ((clrmask & (IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK)) != 0) /* If one goes, both go. */ + clrmask |= (IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK); + + spin_lock_irqsave_hw(&__adeos_pipelock,flags); + + adp_current->irqs[irq].control &= ~clrmask; + adp_current->irqs[irq].control |= setmask; + + if ((setmask & IPIPE_ENABLE_MASK) != 0) + desc->handler->enable(irq); + else if ((clrmask & IPIPE_ENABLE_MASK) != 0) + desc->handler->disable(irq); + + spin_unlock_irqrestore_hw(&__adeos_pipelock,flags); + + return 0; +} + +#ifdef CONFIG_ADEOS_THREADS + +asmlinkage static void __adeos_domain_trampoline (void (*entry)(int), int iflag) +/* asmlinkage is there just in case CONFIG_REGPARM is enabled... */ +{ + unsigned long flags; + adeos_declare_cpuid; + + adeos_get_cpu(flags); + clear_bit(IPIPE_SLEEP_FLAG,&adp_current->cpudata[cpuid].status); + adeos_put_cpu(flags); + entry(iflag); +} + +void __adeos_init_domain (adomain_t *adp, adattr_t *attr) + +{ + int estacksz = attr->estacksz > 0 ? attr->estacksz : 8192, _cpuid; + int nr_cpus = num_online_cpus(); + adeos_declare_cpuid; + + /* Here we don't care if a CPU migration occurs since we do not + use the cpuid for accessing per-CPU data, but we don't want + more than one CPU to be passed iflag == 1. */ + + adeos_load_cpuid(); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + { + int **psp = &adp->esp[_cpuid]; + + adp->estackbase[_cpuid] = (int *)kmalloc(estacksz,GFP_KERNEL); + + if (adp->estackbase[_cpuid] == NULL) + panic("Adeos: No memory for domain stack on CPU #%d",_cpuid); + + adp->esp[_cpuid] = adp->estackbase[_cpuid]; + **psp = 0; + *psp = (int *)(((unsigned long)*psp + estacksz - 0x10) & ~0xf); + *--(*psp) = (_cpuid == cpuid); /* iflag */ + *--(*psp) = (int)attr->entry; + *--(*psp) = 0; + *--(*psp) = (int)&__adeos_domain_trampoline; + } +} + +#else /* !CONFIG_ADEOS_THREADS */ + +void __adeos_init_domain (adomain_t *adp, adattr_t *attr) + +{} + +#endif /* CONFIG_ADEOS_THREADS */ + +void __adeos_cleanup_domain (adomain_t *adp) + +{ + int nr_cpus = num_online_cpus(); + int _cpuid; + + adeos_unstall_pipeline_from(adp); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + { +#ifdef CONFIG_SMP + while (adp->cpudata[_cpuid].irq_pending_hi != 0) + cpu_relax(); + + while (test_bit(IPIPE_XPEND_FLAG,&adp->cpudata[_cpuid].status)) + cpu_relax(); +#endif /* CONFIG_SMP */ + +#ifdef CONFIG_ADEOS_THREADS + if (adp->estackbase[_cpuid] != NULL) + kfree(adp->estackbase[_cpuid]); +#endif /* CONFIG_ADEOS_THREADS */ + } +} + +int adeos_get_sysinfo (adsysinfo_t *info) + +{ + info->ncpus = num_online_cpus(); + info->cpufreq = adeos_cpu_freq(); + info->archdep.tmirq = __adeos_tick_irq; +#ifdef CONFIG_X86_TSC + info->archdep.tmfreq = adeos_cpu_freq(); +#else /* !CONFIG_X86_TSC */ + info->archdep.tmfreq = CLOCK_TICK_RATE; +#endif /* CONFIG_X86_TSC */ + + return 0; +} + +int adeos_tune_timer (unsigned long ns, int flags) + +{ + unsigned hz, latch; + unsigned long x; + + if (flags & ADEOS_RESET_TIMER) + latch = LATCH; + else + { + hz = 1000000000 / ns; + + if (hz < HZ) + return -EINVAL; + + latch = (CLOCK_TICK_RATE + hz/2) / hz; + } + + x = adeos_critical_enter(NULL); /* Sync with all CPUs */ + + /* Shamelessly lifted from init_IRQ() in i8259.c */ + outb_p(0x34,0x43); /* binary, mode 2, LSB/MSB, ch 0 */ + outb_p(latch & 0xff,0x40); /* LSB */ + outb(latch >> 8,0x40); /* MSB */ + + adeos_critical_exit(x); + + return 0; +} + +/* adeos_send_ipi() -- Send a specified service IPI to a set of + processors. */ + +int fastcall adeos_send_ipi (unsigned ipi, cpumask_t cpumask) + +{ +#ifdef CONFIG_SMP + unsigned long flags; + adeos_declare_cpuid; + int self; + + switch (ipi) + { + case ADEOS_SERVICE_IPI0: + case ADEOS_SERVICE_IPI1: + case ADEOS_SERVICE_IPI2: + case ADEOS_SERVICE_IPI3: + + break; + + default: + + return -EINVAL; + } + + adeos_lock_cpu(flags); + + self = cpu_isset(cpuid,cpumask); + cpu_clear(cpuid,cpumask); + + if (!cpus_empty(cpumask)) + send_IPI_mask(cpumask,ipi + FIRST_EXTERNAL_VECTOR); + + if (self) + adeos_trigger_irq(ipi); + + adeos_unlock_cpu(flags); + +#endif /* CONFIG_SMP */ + + return 0; +} diff -uNrp linux-2.6.13/arch/i386/Kconfig linux-2.6.13-adeos/arch/i386/Kconfig --- linux-2.6.13/arch/i386/Kconfig 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/Kconfig 2005-09-04 12:38:01.000000000 +0200 @@ -980,6 +980,7 @@ config CRASH_DUMP Generate crash dump after being started by kexec. endmenu +source "adeos/Kconfig" menu "Power management options (ACPI, APM)" depends on !X86_VOYAGER diff -uNrp linux-2.6.13/arch/i386/kernel/Makefile linux-2.6.13-adeos/arch/i386/kernel/Makefile --- linux-2.6.13/arch/i386/kernel/Makefile 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/Makefile 2005-09-04 12:38:02.000000000 +0200 @@ -11,6 +11,7 @@ obj-y := process.o semaphore.o signal.o obj-y += cpu/ obj-y += timers/ +obj-$(CONFIG_ADEOS_CORE) += adeos.o obj-$(CONFIG_ACPI_BOOT) += acpi/ obj-$(CONFIG_X86_BIOS_REBOOT) += reboot.o obj-$(CONFIG_MCA) += mca.o diff -uNrp linux-2.6.13/arch/i386/kernel/adeos.c linux-2.6.13-adeos/arch/i386/kernel/adeos.c --- linux-2.6.13/arch/i386/kernel/adeos.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/arch/i386/kernel/adeos.c 2005-10-06 15:57:16.000000000 +0200 @@ -0,0 +1,797 @@ +/* + * linux/arch/i386/kernel/adeos.c + * + * Copyright (C) 2002 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-dependent ADEOS core support for x86. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef CONFIG_X86_LOCAL_APIC +#include +#include +#include +#ifdef CONFIG_X86_IO_APIC +#include +#endif /* CONFIG_X86_IO_APIC */ +#include +#include +#endif /* CONFIG_X86_LOCAL_APIC */ + +struct pt_regs __adeos_tick_regs[ADEOS_NR_CPUS]; + +int __adeos_tick_irq; + +#ifdef CONFIG_X86_LOCAL_APIC +/* Ugly hack allowing 1) to run portions of early kernel init code + which make use of smp_processor_id(), 2) to run SMP kernels on UP + boxen without APIC. See asm-i386/adeos.h */ +int __adeos_apic_mapped; +#endif /* CONFIG_X86_LOCAL_APIC */ + +#ifdef CONFIG_SMP + +volatile u8 __adeos_apicid_2_cpuid[MAX_APICID]; + +static cpumask_t __adeos_cpu_sync_map; + +static cpumask_t __adeos_cpu_lock_map; + +static raw_spinlock_t __adeos_cpu_barrier = RAW_SPIN_LOCK_UNLOCKED; + +static atomic_t __adeos_critical_count = ATOMIC_INIT(0); + +static void (*__adeos_cpu_sync)(void); + +#endif /* CONFIG_SMP */ + +#define __adeos_call_asm_irq_handler(adp,irq) \ + __asm__ __volatile__ ("pushfl\n\t" \ + "push %%cs\n\t" \ + "call *%1\n" \ + : /* no output */ \ + : "a" (irq), "m" ((adp)->irqs[irq].handler)) + +/* Since 2.6, ret_from_intr might identify a need for rescheduling + (raised by the C handler) even from kernel space if the preemption + is enabled, so we should branch to this routine on our return + path. Native (i.e. ASM) handlers do the same. */ + +#define __adeos_call_c_root_irq_handler(adp,irq) \ + __asm__ __volatile__ ("pushfl\n\t" \ + "pushl %%cs\n\t" \ + "pushl $1f\n\t" \ + "pushl $-1\n\t" /* Negative (fake) orig_eax. */ \ + "pushl %%es\n\t" \ + "pushl %%ds\n\t" \ + "pushl %%eax\n\t" \ + "pushl %%ebp\n\t" \ + "pushl %%edi\n\t" \ + "pushl %%esi\n\t" \ + "pushl %%edx\n\t" \ + "pushl %%ecx\n\t" \ + "pushl %%ebx\n\t" \ + "pushl %%eax\n\t" \ + "call *%1\n\t" \ + "addl $4,%%esp\n\t" \ + "jmp ret_from_intr\n\t" \ + "1:\n" \ + : /* no output */ \ + : "a" (irq), "m" ((adp)->irqs[irq].handler)) + +/* Do _not_ forcibly re-enable hw IRQs in the following trampoline + used for non-root domains; unlike Linux handlers, non-root domain + handlers are fully in control of the hw masking state. */ + +#define __adeos_call_c_irq_handler(adp,irq) \ + __asm__ __volatile__ ("pushl %%ebp\n\t" \ + "pushl %%edi\n\t" \ + "pushl %%esi\n\t" \ + "pushl %%edx\n\t" \ + "pushl %%ecx\n\t" \ + "pushl %%ebx\n\t" \ + "pushl %%eax\n\t" \ + "call *%1\n\t" \ + "addl $4,%%esp\n\t" \ + "popl %%ebx\n\t" \ + "popl %%ecx\n\t" \ + "popl %%edx\n\t" \ + "popl %%esi\n\t" \ + "popl %%edi\n\t" \ + "popl %%ebp\n" \ + : /* no output */ \ + : "a" (irq), "m" ((adp)->irqs[irq].handler)) + +static __inline__ unsigned long flnz (unsigned long word) { + __asm__("bsrl %1, %0" + : "=r" (word) + : "r" (word)); + return word; +} + +int __adeos_ack_system_irq (unsigned irq) { +#ifdef CONFIG_X86_LOCAL_APIC + __ack_APIC_irq(); /* Takes no spin lock, right? */ +#endif /* CONFIG_X86_LOCAL_APIC */ + return 1; +} + +#ifdef CONFIG_SMP + +/* Always called with hw interrupts off. */ + +static void __adeos_do_critical_sync (unsigned irq) + +{ + adeos_declare_cpuid; + + adeos_load_cpuid(); + + cpu_set(cpuid,__adeos_cpu_sync_map); + + /* Now we are in sync with the lock requestor running on another + CPU. Enter a spinning wait until he releases the global + lock. */ + adeos_spin_lock(&__adeos_cpu_barrier); + + /* Got it. Now get out. */ + + if (__adeos_cpu_sync) + /* Call the sync routine if any. */ + __adeos_cpu_sync(); + + adeos_spin_unlock(&__adeos_cpu_barrier); + + cpu_clear(cpuid,__adeos_cpu_sync_map); +} + +int __adeos_hw_cpuid (void) + +{ + unsigned long flags; + int cpuid; + + if (!__adeos_apic_mapped) + return 0; + + adeos_hw_local_irq_save(flags); + cpuid = __adeos_apicid_2_cpuid[adeos_smp_apic_id()]; + adeos_hw_local_irq_restore(flags); + + return cpuid; +} + +#endif /* CONFIG_SMP */ + +/* adeos_critical_enter() -- Grab the superlock excluding all CPUs + but the current one from a critical section. This lock is used when + we must enforce a global critical section for a single CPU in a + possibly SMP system whichever context the CPUs are running + (i.e. interrupt handler or regular thread). */ + +unsigned long adeos_critical_enter (void (*syncfn)(void)) + +{ + unsigned long flags; + + adeos_hw_local_irq_save(flags); + +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) /* We might be running a SMP-kernel on a UP box... */ + { + adeos_declare_cpuid; + cpumask_t lock_map; + + adeos_load_cpuid(); + + if (!cpu_test_and_set(cpuid,__adeos_cpu_lock_map)) + { + while (cpu_test_and_set(BITS_PER_LONG - 1,__adeos_cpu_lock_map)) + { + /* Refer to the explanations found in + linux/arch/asm-i386/irq.c about + SUSPECTED_CPU_OR_CHIPSET_BUG_WORKAROUND for more about + this strange loop. */ + int n = 0; + do { cpu_relax(); } while (++n < cpuid); + } + + adeos_spin_lock(&__adeos_cpu_barrier); + + __adeos_cpu_sync = syncfn; + + /* Send the sync IPI to all processors but the current one. */ + send_IPI_allbutself(ADEOS_CRITICAL_VECTOR); + +#ifdef cpus_andnot + cpus_andnot(lock_map,cpu_online_map,__adeos_cpu_lock_map); +#else /* !cpus_andnot */ + { + cpumask_t not_lock_map = __adeos_cpu_lock_map; + cpus_complement(not_lock_map); + cpus_and(lock_map,cpu_online_map,not_lock_map); + } +#endif /* cpus_andnot */ + + while (!cpus_equal(__adeos_cpu_sync_map,lock_map)) + cpu_relax(); + } + + atomic_inc(&__adeos_critical_count); + } +#endif /* CONFIG_SMP */ + + return flags; +} + +/* adeos_critical_exit() -- Release the superlock. */ + +void adeos_critical_exit (unsigned long flags) + +{ +#ifdef CONFIG_SMP + if (num_online_cpus() > 1) /* We might be running a SMP-kernel on a UP box... */ + { + adeos_declare_cpuid; + + adeos_load_cpuid(); + + if (atomic_dec_and_test(&__adeos_critical_count)) + { + adeos_spin_unlock(&__adeos_cpu_barrier); + + while (!cpus_empty(__adeos_cpu_sync_map)) + cpu_relax(); + + cpu_clear(cpuid,__adeos_cpu_lock_map); + cpu_clear(BITS_PER_LONG - 1,__adeos_cpu_lock_map); + } + } +#endif /* CONFIG_SMP */ + + adeos_hw_local_irq_restore(flags); +} + +void __adeos_init_stage (adomain_t *adp) + +{ + int cpuid, n; + + for (cpuid = 0; cpuid < ADEOS_NR_CPUS; cpuid++) + { + adp->cpudata[cpuid].irq_pending_hi = 0; + + for (n = 0; n < IPIPE_IRQ_IWORDS; n++) + adp->cpudata[cpuid].irq_pending_lo[n] = 0; + + for (n = 0; n < IPIPE_NR_IRQS; n++) + adp->cpudata[cpuid].irq_hits[n] = 0; + } + + for (n = 0; n < IPIPE_NR_IRQS; n++) + { + adp->irqs[n].acknowledge = NULL; + adp->irqs[n].handler = NULL; + adp->irqs[n].control = IPIPE_PASS_MASK; /* Pass but don't handle */ + } + +#ifdef CONFIG_SMP + adp->irqs[ADEOS_CRITICAL_IPI].acknowledge = &__adeos_ack_system_irq; + adp->irqs[ADEOS_CRITICAL_IPI].handler = &__adeos_do_critical_sync; + /* Immediately handle in the current domain but *never* pass */ + adp->irqs[ADEOS_CRITICAL_IPI].control = IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SYSTEM_MASK; +#endif /* CONFIG_SMP */ +} + +/* __adeos_sync_stage() -- Flush the pending IRQs for the current + domain (and processor). This routine flushes the interrupt log + (see "Optimistic interrupt protection" from D. Stodolsky et al. for + more on the deferred interrupt scheme). Every interrupt that + occurred while the pipeline was stalled gets played. WARNING: + callers on SMP boxen should always check for CPU migration on + return of this routine. One can control the kind of interrupts + which are going to be sync'ed using the syncmask + parameter. IPIPE_IRQMASK_ANY plays them all, IPIPE_IRQMASK_VIRT + plays virtual interrupts only. This routine must be called with hw + interrupts off. */ + +void fastcall __adeos_sync_stage (unsigned long syncmask) + +{ + unsigned long mask, submask; + struct adcpudata *cpudata; + adeos_declare_cpuid; + int level, rank; + adomain_t *adp; + unsigned irq; + + adeos_load_cpuid(); + adp = adp_cpu_current[cpuid]; + cpudata = &adp->cpudata[cpuid]; + + if (__test_and_set_bit(IPIPE_SYNC_FLAG,&cpudata->status)) + return; + + /* The policy here is to keep the dispatching code interrupt-free + by stalling the current stage. If the upper domain handler + (which we call) wants to re-enable interrupts while in a safe + portion of the code (e.g. SA_INTERRUPT flag unset for Linux's + sigaction()), it will have to unstall (then stall again before + returning to us!) the stage when it sees fit. */ + + while ((mask = (cpudata->irq_pending_hi & syncmask)) != 0) + { + /* Give a slight priority advantage to high-numbered IRQs + like the virtual ones. */ + level = flnz(mask); + __clear_bit(level,&cpudata->irq_pending_hi); + + while ((submask = cpudata->irq_pending_lo[level]) != 0) + { + rank = flnz(submask); + irq = (level << IPIPE_IRQ_ISHIFT) + rank; + + if (test_bit(IPIPE_LOCK_FLAG,&adp->irqs[irq].control)) + { + __clear_bit(rank,&cpudata->irq_pending_lo[level]); + continue; + } + + if (--cpudata->irq_hits[irq] == 0) + __clear_bit(rank,&cpudata->irq_pending_lo[level]); + + __set_bit(IPIPE_STALL_FLAG,&cpudata->status); + +#ifdef CONFIG_ADEOS_PROFILING + __adeos_profile_data[cpuid].irqs[irq].n_synced++; + adeos_hw_tsc(__adeos_profile_data[cpuid].irqs[irq].t_synced); +#endif /* CONFIG_ADEOS_PROFILING */ + + if (adp == adp_root) + { + /* Make sure to re-enable hw interrupts to reduce + preemption latency by higher priority domains when + calling the Linux handlers in the next two + trampolines. Additionally, this ensures that the + forged interrupt frame will allow the final check + for a rescheduling opportunity in ret_from_intr. */ + + adeos_hw_sti(); + + if (test_bit(IPIPE_CALLASM_FLAG,&adp->irqs[irq].control)) + __adeos_call_asm_irq_handler(adp,irq); + else + { + irq_enter(); + __adeos_call_c_root_irq_handler(adp,irq); + irq_exit(); + } + + adeos_hw_cli(); + } + else + { + __clear_bit(IPIPE_SYNC_FLAG,&cpudata->status); + __adeos_call_c_irq_handler(adp,irq); + __set_bit(IPIPE_SYNC_FLAG,&cpudata->status); + } + +#ifdef CONFIG_SMP + { + int _cpuid = adeos_processor_id(); + + if (_cpuid != cpuid) /* Handle CPU migration. */ + { + /* We expect any domain to clear the SYNC bit each + time it switches in a new task, so that preemptions + and/or CPU migrations (in the SMP case) over the + ISR do not lock out the log syncer for some + indefinite amount of time. In the Linux case, + schedule() handles this (see kernel/sched.c). For + this reason, we don't bother clearing it here for + the source CPU in the migration handling case, + since it must have scheduled another task in by + now. */ + cpuid = _cpuid; + cpudata = &adp->cpudata[cpuid]; + __set_bit(IPIPE_SYNC_FLAG,&cpudata->status); + } + } +#endif /* CONFIG_SMP */ + + __clear_bit(IPIPE_STALL_FLAG,&cpudata->status); + } + } + + __clear_bit(IPIPE_SYNC_FLAG,&cpudata->status); +} + +/* __adeos_walk_pipeline(): Must be called with local interrupts + disabled. */ + +static inline void __adeos_walk_pipeline (struct list_head *pos, int cpuid) + +{ + adomain_t *this_domain = adp_cpu_current[cpuid]; + + while (pos != &__adeos_pipeline) + { + adomain_t *next_domain = list_entry(pos,adomain_t,p_link); + + if (test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) + break; /* Stalled stage -- do not go further. */ + + if (next_domain->cpudata[cpuid].irq_pending_hi != 0) + { + /* Since the critical IPI might be dispatched by the + following actions, the current domain might not be + linked to the pipeline anymore after its handler + returns on SMP boxes, even if the domain remains valid + (see adeos_unregister_domain()), so don't make any + dangerous assumptions here. */ + + if (next_domain == this_domain) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + else + { + __adeos_switch_to(this_domain,next_domain,cpuid); + + adeos_load_cpuid(); /* Processor might have changed. */ + + if (this_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&this_domain->cpudata[cpuid].status)) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + } + + break; + } + else if (next_domain == this_domain) + break; + + pos = next_domain->p_link.next; + } +} + +/* __adeos_handle_irq() -- ADEOS's generic IRQ handler. An optimistic + interrupt protection log is maintained here for each domain. + Interrupts are off on entry. */ + +int __adeos_handle_irq (struct pt_regs regs) + +{ + unsigned irq = regs.orig_eax; + struct list_head *head, *pos; + adomain_t *this_domain; + adeos_declare_cpuid; + int m_ack, s_ack; + + if (regs.orig_eax < 0) + { + irq &= 0xff; + m_ack = 0; + } + else + m_ack = 1; + + adeos_load_cpuid(); + + this_domain = adp_cpu_current[cpuid]; + +#ifdef CONFIG_ADEOS_PROFILING + __adeos_profile_data[cpuid].irqs[irq].n_handled++; + adeos_hw_tsc(__adeos_profile_data[cpuid].irqs[irq].t_handled); +#endif /* CONFIG_ADEOS_PROFILING */ + + s_ack = m_ack; + + if (test_bit(IPIPE_STICKY_FLAG,&this_domain->irqs[irq].control)) + head = &this_domain->p_link; + else + head = __adeos_pipeline.next; + + /* Ack the interrupt. */ + + pos = head; + + while (pos != &__adeos_pipeline) + { + adomain_t *next_domain = list_entry(pos,adomain_t,p_link); + + /* For each domain handling the incoming IRQ, mark it as + pending in its log. */ + + if (test_bit(IPIPE_HANDLE_FLAG,&next_domain->irqs[irq].control)) + { + /* Domains that handle this IRQ are polled for + acknowledging it by decreasing priority order. The + interrupt must be made pending _first_ in the domain's + status flags before the PIC is unlocked. */ + + next_domain->cpudata[cpuid].irq_hits[irq]++; + __adeos_set_irq_bit(next_domain,cpuid,irq); + + /* Always get the first master acknowledge available. Once + we've got it, allow slave acknowledge handlers to run + (until one of them stops us). */ + + if (!m_ack) + m_ack = next_domain->irqs[irq].acknowledge(irq); + else if (test_bit(IPIPE_SHARED_FLAG,&next_domain->irqs[irq].control) && !s_ack) + s_ack = next_domain->irqs[irq].acknowledge(irq); + } + + /* If the domain does not want the IRQ to be passed down the + interrupt pipe, exit the loop now. */ + + if (!test_bit(IPIPE_PASS_FLAG,&next_domain->irqs[irq].control)) + break; + + pos = next_domain->p_link.next; + } + + if (irq == __adeos_tick_irq) + { + __adeos_tick_regs[cpuid].eflags = regs.eflags; + __adeos_tick_regs[cpuid].eip = regs.eip; + __adeos_tick_regs[cpuid].xcs = regs.xcs; +#if defined(CONFIG_SMP) && defined(CONFIG_FRAME_POINTER) + /* Linux profiling code needs this. */ + __adeos_tick_regs[cpuid].ebp = regs.ebp; +#endif /* CONFIG_SMP && CONFIG_FRAME_POINTER */ + if (__adeos_pipeline_head_p(adp_root) && + adp_root->cpudata[cpuid].irq_hits[irq] > 1) + /* Emulate a loss of clock ticks if Linux is owning the + * time source. The drift will be compensated by the timer + * support code. */ + adp_root->cpudata[cpuid].irq_hits[irq] = 1; + } + + /* Now walk the pipeline, yielding control to the highest priority + domain that has pending interrupt(s) or immediately to the + current domain if the interrupt has been marked as + 'sticky'. This search does not go beyond the current domain in + the pipeline. To understand this code properly, one must keep + in mind that domains having a higher priority than the current + one are sleeping on the adeos_suspend_domain() service. In + addition, domains having a lower priority have been preempted + by an interrupt dispatched to a higher priority domain. Once + the first and highest priority stage has been selected here, + the subsequent stages will be activated in turn when each + visited domain calls adeos_suspend_domain() to wake up its + neighbour down the pipeline. */ + + __adeos_walk_pipeline(head,cpuid); + + adeos_load_cpuid(); + + if (adp_cpu_current[cpuid] != adp_root || + test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status)) + return 0; + +#ifdef CONFIG_SMP + /* Prevent a spurious rescheduling from being triggered on + preemptible kernels along the way out through ret_from_intr. */ + if (regs.orig_eax < 0) + __set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); +#endif /* CONFIG_SMP */ + + return 1; +} + +/* adeos_trigger_irq() -- Push the interrupt to the pipeline entry + just like if it has been actually received from a hw source. This + both works for real and virtual interrupts. This also means that + the current domain might be immediately preempted by a higher + priority domain who happens to handle this interrupt. */ + +int fastcall adeos_trigger_irq (unsigned irq) + +{ + struct pt_regs regs; + unsigned long flags; + + if (irq >= IPIPE_NR_IRQS || + (adeos_virtual_irq_p(irq) && + !test_bit(irq - IPIPE_VIRQ_BASE,&__adeos_virtual_irq_map))) + return -EINVAL; + + adeos_hw_local_irq_save(flags); + + regs.orig_eax = irq; /* Won't be acked */ + regs.xcs = __KERNEL_CS; + regs.eflags = flags; + + __adeos_handle_irq(regs); + + adeos_hw_local_irq_restore(flags); + + return 1; +} + +static inline void __fixup_if (struct pt_regs *regs) + +{ + adeos_declare_cpuid; + unsigned long flags; + + adeos_get_cpu(flags); + + if (adp_cpu_current[cpuid] == adp_root) + { + /* Have the saved hw state look like the domain stall bit, so + that __adeos_unstall_iret_root() restores the proper + pipeline state for the root stage upon exit. */ + + if (test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status)) + regs->eflags &= ~0x200; + else + regs->eflags |= 0x200; + } + + adeos_put_cpu(flags); +} + +asmlinkage void __adeos_if_fixup_root (struct pt_regs regs) + +{ + if (adp_pipelined) + __fixup_if(®s); +} + +/* Check the interrupt flag to make sure the existing preemption + opportunity upon in-kernel resumption could be exploited. If + pipelining is active, the stall bit of the root domain is checked, + otherwise, the EFLAGS register from the stacked interrupt frame is + tested. In case a rescheduling could take place in pipelined mode, + the root stage is stalled before the hw interrupts are + re-enabled. This routine must be called with hw interrupts off. */ + +asmlinkage int __adeos_kpreempt_root (struct pt_regs regs) + +{ + adeos_declare_cpuid; + unsigned long flags; + + if (!adp_pipelined) + return !!(regs.eflags & 0x200); + + adeos_get_cpu(flags); + + if (test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status)) + { + adeos_put_cpu(flags); + return 0; /* Root stage is stalled: rescheduling denied. */ + } + + __adeos_stall_root(); + adeos_hw_sti(); + + return 1; /* Ok, may reschedule now. */ +} + +asmlinkage int __adeos_enter_syscall (struct pt_regs regs) + +{ + adeos_declare_cpuid; + unsigned long flags; + + /* This routine either returns: + 0 -- if the syscall is to be passed to Linux; + 1 -- if the syscall should not be passed to Linux, and no + tail work should be performed; + -1 -- if the syscall should not be passed to Linux but the + tail work has to be performed. */ + + if (__adeos_event_monitors[ADEOS_SYSCALL_PROLOGUE] > 0 && + __adeos_handle_event(ADEOS_SYSCALL_PROLOGUE,®s) > 0) + { + /* We might enter here over a non-root domain and exit over + the root one as a result of the issued syscall (i.e. by + recycling the register set of the current context across + the migration), so we need to fixup the interrupt flag upon + return too, so that __adeos_unstall_iret_root() resets the + correct stall bit on exit. */ + __fixup_if(®s); + + if (adp_current == adp_root && !in_atomic()) + { + /* Sync pending VIRQs before _TIF_NEED_RESCHED is + * tested. */ + + adeos_lock_cpu(flags); + + if ((adp_root->cpudata[cpuid].irq_pending_hi & IPIPE_IRQMASK_VIRT) != 0) + __adeos_sync_stage(IPIPE_IRQMASK_VIRT); + + adeos_unlock_cpu(flags); + + return -1; + } + + return 1; + } + + return 0; +} + +asmlinkage int __adeos_exit_syscall (void) + +{ + if (__adeos_event_monitors[ADEOS_SYSCALL_EPILOGUE] > 0) + return __adeos_handle_event(ADEOS_SYSCALL_EPILOGUE,NULL); + + return 0; +} + +asmlinkage void __adeos_unstall_iret_root (struct pt_regs regs) + +{ + adeos_declare_cpuid; + + if (!adp_pipelined) + return; + + /* Emulate a software IRET. */ + + adeos_hw_cli(); + + adeos_load_cpuid(); + + /* Restore the software state as it used to be on kernel entry. */ + + if (!(regs.eflags & 0x200)) + { + __set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + regs.eflags |= 0x200; + } + else + { + __clear_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + + /* Only sync virtual IRQs here, so that we don't recurse + indefinitely in case of an external interrupt flood. */ + + if ((adp_root->cpudata[cpuid].irq_pending_hi & IPIPE_IRQMASK_VIRT) != 0) + __adeos_sync_stage(IPIPE_IRQMASK_VIRT); + } +} + +EXPORT_SYMBOL(__adeos_init_stage); +EXPORT_SYMBOL(__adeos_sync_stage); +EXPORT_SYMBOL(__adeos_ack_system_irq); +EXPORT_SYMBOL(__adeos_handle_irq); +#ifdef CONFIG_SMP +EXPORT_SYMBOL(__adeos_hw_cpuid); +#endif /* CONFIG_SMP */ +EXPORT_SYMBOL(__adeos_tick_irq); +EXPORT_SYMBOL(adeos_critical_enter); +EXPORT_SYMBOL(adeos_critical_exit); +EXPORT_SYMBOL(adeos_trigger_irq); diff -uNrp linux-2.6.13/arch/i386/kernel/apic.c linux-2.6.13-adeos/arch/i386/kernel/apic.c --- linux-2.6.13/arch/i386/kernel/apic.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/apic.c 2005-09-04 12:38:01.000000000 +0200 @@ -69,7 +69,7 @@ void ack_bad_irq(unsigned int irq) * unexpected vectors occur) that might lock up the APIC * completely. */ - ack_APIC_irq(); + __ack_APIC_irq(); } void __init apic_intr_init(void) @@ -818,6 +818,9 @@ void __init init_apic_mappings(void) apic_phys = mp_lapic_addr; set_fixmap_nocache(FIX_APIC_BASE, apic_phys); +#ifdef CONFIG_ADEOS_CORE + __adeos_apic_mapped = 1; +#endif /* CONFIG_ADEOS_CORE */ printk(KERN_DEBUG "mapped APIC to %08lx (%08lx)\n", APIC_BASE, apic_phys); @@ -1179,6 +1182,9 @@ inline void smp_local_timer_interrupt(st fastcall void smp_apic_timer_interrupt(struct pt_regs *regs) { int cpu = smp_processor_id(); +#ifdef CONFIG_ADEOS_CORE + struct pt_regs *_regs = adp_pipelined ? __adeos_tick_regs + cpu : regs; +#endif /* CONFIG_ADEOS_CORE */ /* * the NMI deadlock-detector uses this. @@ -1196,7 +1202,11 @@ fastcall void smp_apic_timer_interrupt(s * interrupt lock, which is the WrongThing (tm) to do. */ irq_enter(); +#ifdef CONFIG_ADEOS_CORE + smp_local_timer_interrupt(_regs); +#else /* !CONFIG_ADEOS_CORE */ smp_local_timer_interrupt(regs); +#endif /* CONFIG_ADEOS_CORE */ irq_exit(); } @@ -1215,7 +1225,7 @@ fastcall void smp_spurious_interrupt(str */ v = apic_read(APIC_ISR + ((SPURIOUS_APIC_VECTOR & ~0x1f) >> 1)); if (v & (1 << (SPURIOUS_APIC_VECTOR & 0x1f))) - ack_APIC_irq(); + __ack_APIC_irq(); /* see sw-dev-man vol 3, chapter 7.4.13.5 */ printk(KERN_INFO "spurious APIC interrupt on CPU#%d, should never happen.\n", diff -uNrp linux-2.6.13/arch/i386/kernel/cpu/mcheck/p4.c linux-2.6.13-adeos/arch/i386/kernel/cpu/mcheck/p4.c --- linux-2.6.13/arch/i386/kernel/cpu/mcheck/p4.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/cpu/mcheck/p4.c 2005-09-04 12:38:01.000000000 +0200 @@ -50,6 +50,9 @@ static void intel_thermal_interrupt(stru unsigned int cpu = smp_processor_id(); static unsigned long next[NR_CPUS]; +#ifdef CONFIG_ADEOS_CORE + if (!adp_pipelined) +#endif /* CONFIG_ADEOS_CORE */ ack_APIC_irq(); if (time_after(next[cpu], jiffies)) diff -uNrp linux-2.6.13/arch/i386/kernel/cpu/mtrr/cyrix.c linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/cyrix.c --- linux-2.6.13/arch/i386/kernel/cpu/mtrr/cyrix.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/cyrix.c 2005-09-04 12:38:01.000000000 +0200 @@ -118,6 +118,11 @@ static void prepare_set(void) { u32 cr0; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ + /* Save value of CR4 and clear Page Global Enable (bit 7) */ if ( cpu_has_pge ) { cr4 = read_cr4(); @@ -137,10 +142,18 @@ static void prepare_set(void) /* Cyrix ARRs - everything else were excluded at the top */ setCx86(CX86_CCR3, (ccr3 & 0x0f) | 0x10); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } static void post_set(void) { +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ + /* Flush caches and TLBs */ wbinvd(); @@ -153,6 +166,10 @@ static void post_set(void) /* Restore value of CR4 */ if ( cpu_has_pge ) write_cr4(cr4); + +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } static void cyrix_set_arr(unsigned int reg, unsigned long base, diff -uNrp linux-2.6.13/arch/i386/kernel/cpu/mtrr/generic.c linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/generic.c --- linux-2.6.13/arch/i386/kernel/cpu/mtrr/generic.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/generic.c 2005-09-04 12:38:01.000000000 +0200 @@ -253,6 +253,11 @@ static void prepare_set(void) spin_lock(&set_atomicity_lock); +#ifdef CONFIG_ADEOS_CORE + { + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ /* Enter the no-fill (CD=1, NW=0) cache mode and flush caches. */ cr0 = read_cr0() | 0x40000000; /* set CD flag */ write_cr0(cr0); @@ -272,10 +277,18 @@ static void prepare_set(void) /* Disable MTRRs, and set the default type to uncached */ mtrr_wrmsr(MTRRdefType_MSR, deftype_lo & 0xf300UL, deftype_hi); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); + } +#endif /* CONFIG_ADEOS_CORE */ } static void post_set(void) { +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ /* Flush TLBs (no need to flush caches - they are disabled) */ __flush_tlb(); @@ -289,6 +302,9 @@ static void post_set(void) if ( cpu_has_pge ) write_cr4(cr4); spin_unlock(&set_atomicity_lock); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } static void generic_set_all(void) diff -uNrp linux-2.6.13/arch/i386/kernel/cpu/mtrr/state.c linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/state.c --- linux-2.6.13/arch/i386/kernel/cpu/mtrr/state.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/cpu/mtrr/state.c 2005-09-04 12:38:01.000000000 +0200 @@ -12,7 +12,11 @@ void set_mtrr_prepare_save(struct set_mt unsigned int cr0; /* Disable interrupts locally */ +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_save(ctxt->flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_save(ctxt->flags); +#endif /* CONFIG_ADEOS_CORE */ if (use_intel() || is_cpu(CYRIX)) { @@ -73,6 +77,10 @@ void set_mtrr_done(struct set_mtrr_conte write_cr4(ctxt->cr4val); } /* Re-enable interrupts locally (if enabled previously) */ +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(ctxt->flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_restore(ctxt->flags); +#endif /* CONFIG_ADEOS_CORE */ } diff -uNrp linux-2.6.13/arch/i386/kernel/entry.S linux-2.6.13-adeos/arch/i386/kernel/entry.S --- linux-2.6.13/arch/i386/kernel/entry.S 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/entry.S 2005-10-01 21:55:55.000000000 +0200 @@ -82,6 +82,14 @@ VM_MASK = 0x00020000 #define resume_kernel restore_nocheck #endif +#ifdef CONFIG_ADEOS_CORE +#ifdef CONFIG_PREEMPT +#undef preempt_stop +#define preempt_stop call __adeos_stall_root +#endif +#define restore_nocheck .adeos_unstall_and_restore_nocheck +#endif /* CONFIG_ADEOS_CORE */ + #define SAVE_ALL \ cld; \ pushl %es; \ @@ -141,14 +149,22 @@ ENTRY(ret_from_fork) ALIGN ret_from_exception: preempt_stop +#ifdef CONFIG_ADEOS_CORE +ENTRY(ret_from_intr) +#else /* !CONFIG_ADEOS_CORE */ ret_from_intr: +#endif /* CONFIG_ADEOS_CORE */ GET_THREAD_INFO(%ebp) movl EFLAGS(%esp), %eax # mix EFLAGS and CS movb CS(%esp), %al testl $(VM_MASK | 3), %eax jz resume_kernel ENTRY(resume_userspace) - cli # make sure we don't miss an interrupt +#ifdef CONFIG_ADEOS_CORE + call __adeos_stall_root +#else /* !CONFIG_ADEOS_CORE */ + cli # make sure we don't miss an interrupt +#endif /* CONFIG_ADEOS_CORE */ # setting need_resched or sigpending # between sampling and the iret movl TI_flags(%ebp), %ecx @@ -159,14 +175,23 @@ ENTRY(resume_userspace) #ifdef CONFIG_PREEMPT ENTRY(resume_kernel) - cli +#ifdef CONFIG_ADEOS_CORE + call __adeos_stall_root +#else /* !CONFIG_ADEOS_CORE */ + cli # make sure we don't miss an interrupt +#endif /* CONFIG_ADEOS_CORE */ cmpl $0,TI_preempt_count(%ebp) # non-zero preempt_count ? jnz restore_nocheck need_resched: movl TI_flags(%ebp), %ecx # need_resched set ? testb $_TIF_NEED_RESCHED, %cl jz restore_all - testl $IF_MASK,EFLAGS(%esp) # interrupts off (exception path) ? +#ifdef CONFIG_ADEOS_CORE + call __adeos_kpreempt_root # prepare for kernel preemption opportunity + testl %eax,%eax +#else /* !CONFIG_ADEOS_CORE */ + testl $IF_MASK,EFLAGS(%esp) # interrupts off (exception path) ? +#endif /* CONFIG_ADEOS_CORE */ jz restore_all call preempt_schedule_irq jmp need_resched @@ -201,6 +226,14 @@ sysenter_past_esp: pushl %eax SAVE_ALL GET_THREAD_INFO(%ebp) +#ifdef CONFIG_ADEOS_CORE + call __adeos_if_fixup_root + call __adeos_enter_syscall + testl %eax,%eax + js .adeos_syswork + jne .adeos_sysdone + movl ORIG_EAX(%esp),%eax +#endif /* CONFIG_ADEOS_CORE */ /* Note, _TIF_SECCOMP is bit number 8, and so it needs testw and not testb */ testw $(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SECCOMP),TI_flags(%ebp) @@ -209,11 +242,24 @@ sysenter_past_esp: jae syscall_badsys call *sys_call_table(,%eax,4) movl %eax,EAX(%esp) +#ifdef CONFIG_ADEOS_CORE + call __adeos_exit_syscall + testl %eax,%eax + jne .adeos_sysdone +.adeos_syswork: + call __adeos_stall_root +#else /* !CONFIG_ADEOS_CORE */ cli +#endif /* CONFIG_ADEOS_CORE */ movl TI_flags(%ebp), %ecx testw $_TIF_ALLWORK_MASK, %cx jne syscall_exit_work /* if something modifies registers it must also disable sysexit */ +#ifdef CONFIG_ADEOS_CORE + call __adeos_unstall_iret_root +.adeos_sysdone: + movl EAX(%esp),%eax +#endif /* CONFIG_ADEOS_CORE */ movl EIP(%esp), %edx movl OLDESP(%esp), %ecx xorl %ebp,%ebp @@ -226,6 +272,14 @@ ENTRY(system_call) pushl %eax # save orig_eax SAVE_ALL GET_THREAD_INFO(%ebp) +#ifdef CONFIG_ADEOS_CORE + call __adeos_if_fixup_root + call __adeos_enter_syscall + testl %eax,%eax + js syscall_exit + jne .adeos_restore_nocheck + movl ORIG_EAX(%esp),%eax +#endif /* CONFIG_ADEOS_CORE */ # system call tracing in operation /* Note, _TIF_SECCOMP is bit number 8, and so it needs testw and not testb */ testw $(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SECCOMP),TI_flags(%ebp) @@ -235,8 +289,16 @@ ENTRY(system_call) syscall_call: call *sys_call_table(,%eax,4) movl %eax,EAX(%esp) # store the return value +#ifdef CONFIG_ADEOS_CORE + call __adeos_exit_syscall + testl %eax,%eax + jne .adeos_restore_nocheck +syscall_exit: + call __adeos_stall_root +#else /* !CONFIG_ADEOS_CORE */ syscall_exit: cli # make sure we don't miss an interrupt +#endif /* CONFIG_ADEOS_CORE */ # setting need_resched or sigpending # between sampling and the iret movl TI_flags(%ebp), %ecx @@ -253,7 +315,13 @@ restore_all: andl $(VM_MASK | (4 << 8) | 3), %eax cmpl $((4 << 8) | 3), %eax je ldt_ss # returning to user-space with LDT SS +#ifdef CONFIG_ADEOS_CORE +.adeos_unstall_and_restore_nocheck: + call __adeos_unstall_iret_root +.adeos_restore_nocheck: +#else /* !CONFIG_ADEOS_CORE */ restore_nocheck: +#endif /* CONFIG_ADEOS_CORE */ RESTORE_REGS addl $4, %esp 1: iret @@ -300,8 +368,15 @@ work_pending: testb $_TIF_NEED_RESCHED, %cl jz work_notifysig work_resched: +#ifdef CONFIG_ADEOS_CORE + sti +#endif /* CONFIG_ADEOS_CORE */ call schedule +#ifdef CONFIG_ADEOS_CORE + call __adeos_stall_root +#else /* !CONFIG_ADEOS_CORE */ cli # make sure we don't miss an interrupt +#endif /* CONFIG_ADEOS_CORE */ # setting need_resched or sigpending # between sampling and the iret movl TI_flags(%ebp), %ecx @@ -348,7 +423,11 @@ syscall_trace_entry: syscall_exit_work: testb $(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SINGLESTEP), %cl jz work_pending +#ifdef CONFIG_ADEOS_CORE + call __adeos_unstall_root +#else /* CONFIG_ADEOS_CORE */ sti # could let do_syscall_trace() call +#endif /* CONFIG_ADEOS_CORE */ # schedule() instead movl %esp, %eax movl $1, %edx @@ -399,7 +478,11 @@ ENTRY(interrupt) vector=0 ENTRY(irq_entries_start) +#ifdef CONFIG_ADEOS_CORE +.rept 224 +#else .rept NR_IRQS +#endif ALIGN 1: pushl $vector-256 jmp common_interrupt @@ -427,6 +510,36 @@ ENTRY(name) \ /* The include is where all of the SMP etc. interrupts come from */ #include "entry_arch.h" +#ifdef CONFIG_ADEOS_CORE + +.data +ENTRY(__adeos_irq_trampolines) +.text + +vector=0 +ENTRY(__adeos_irq_entries) +.rept 224 + ALIGN +1: pushl $vector-256 + jmp __adeos_irq_common +.data + .long 1b +.text +vector=vector+1 +.endr + + ALIGN +__adeos_irq_common: + SAVE_ALL + call __adeos_handle_irq + testl %eax,%eax + jnz ret_from_intr + RESTORE_REGS + addl $4, %esp + iret + +#endif /* !CONFIG_ADEOS_CORE */ + ENTRY(divide_error) pushl $0 # no error code pushl $do_divide_error @@ -455,6 +568,9 @@ error_code: movl %ecx, %es movl %esp,%eax # pt_regs pointer call *%edi +#ifdef CONFIG_ADEOS_CORE + call __adeos_if_fixup_root +#endif /* CONFIG_ADEOS_CORE */ jmp ret_from_exception ENTRY(coprocessor_error) @@ -470,6 +586,9 @@ ENTRY(simd_coprocessor_error) ENTRY(device_not_available) pushl $-1 # mark this as an int SAVE_ALL +#ifdef CONFIG_ADEOS_CORE + call __adeos_if_fixup_root +#endif /* CONFIG_ADEOS_CORE */ movl %cr0, %eax testl $0x4, %eax # EM (math emulation bit) jne device_not_available_emulate @@ -549,7 +668,13 @@ nmi_stack_correct: xorl %edx,%edx # zero error code movl %esp,%eax # pt_regs pointer call do_nmi +#ifdef CONFIG_ADEOS_CORE + # FIXME: we need to check for a return to + # user-space on a 16bit stack even in the NMI case + jmp .adeos_restore_nocheck +#else /* !CONFIG_ADEOS_CORE */ jmp restore_all +#endif /* CONFIG_ADEOS_CORE */ nmi_stack_fixup: FIX_STACK(12,nmi_stack_correct, 1) diff -uNrp linux-2.6.13/arch/i386/kernel/i386_ksyms.c linux-2.6.13-adeos/arch/i386/kernel/i386_ksyms.c --- linux-2.6.13/arch/i386/kernel/i386_ksyms.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/i386_ksyms.c 2005-09-04 21:17:36.000000000 +0200 @@ -3,6 +3,51 @@ #include #include +#ifdef CONFIG_ADEOS_CORE +extern int __adeos_irq_trampolines; +EXPORT_SYMBOL(__adeos_irq_trampolines); +#ifdef CONFIG_ADEOS_MODULE +#ifdef CONFIG_X86_LOCAL_APIC +extern int using_apic_timer; +EXPORT_SYMBOL(using_apic_timer); +#ifdef CONFIG_SMP +EXPORT_SYMBOL(__adeos_set_irq_affinity); +#include +EXPORT_SYMBOL(send_IPI_mask_bitmask); +#endif /* CONFIG_SMP */ +#endif /* CONFIG_X86_LOCAL_APIC */ +#endif /* CONFIG_ADEOS_MODULE */ +/* The following are per-platform convenience exports which are needed + by some Adeos domains loaded as kernel modules. */ +#include +extern irq_desc_t irq_desc[]; +EXPORT_SYMBOL(irq_desc); +extern struct desc_struct idt_table[]; +EXPORT_SYMBOL(idt_table); +extern void ret_from_intr(void); +EXPORT_SYMBOL(ret_from_intr); +#ifdef CONFIG_SMP +#include +EXPORT_PER_CPU_SYMBOL(cpu_tlbstate); +#endif /* CONFIG_SMP */ +#if defined(CONFIG_ADEOS_MODULE) && defined(CONFIG_X86_IO_APIC) +EXPORT_SYMBOL(io_apic_irqs); +EXPORT_SYMBOL(irq_vector); +#endif /* CONFIG_ADEOS_MODULE && CONFIG_X86_IO_APIC */ +EXPORT_SYMBOL(set_ldt_desc); +EXPORT_SYMBOL(default_ldt); +EXPORT_PER_CPU_SYMBOL(init_tss); +EXPORT_SYMBOL(__switch_to); +extern void show_stack(struct task_struct *task, unsigned long *esp); +EXPORT_SYMBOL(show_stack); +extern void show_registers(struct pt_regs *regs); +EXPORT_SYMBOL(show_registers); +void show_trace(struct task_struct *task, unsigned long *stack); +EXPORT_SYMBOL(show_trace); +asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long error_code); +EXPORT_SYMBOL(do_page_fault); +#endif /* CONFIG_ADEOS_CORE */ + /* This is definitely a GPL-only symbol */ EXPORT_SYMBOL_GPL(cpu_gdt_table); diff -uNrp linux-2.6.13/arch/i386/kernel/i8259.c linux-2.6.13-adeos/arch/i386/kernel/i8259.c --- linux-2.6.13/arch/i386/kernel/i8259.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/i8259.c 2005-09-04 12:38:02.000000000 +0200 @@ -92,13 +92,14 @@ void disable_8259A_irq(unsigned int irq) unsigned int mask = 1 << irq; unsigned long flags; - spin_lock_irqsave(&i8259A_lock, flags); + spin_lock_irqsave_hw(&i8259A_lock, flags); + pic_irq_lock(irq); cached_irq_mask |= mask; if (irq & 8) outb(cached_slave_mask, PIC_SLAVE_IMR); else outb(cached_master_mask, PIC_MASTER_IMR); - spin_unlock_irqrestore(&i8259A_lock, flags); + spin_unlock_irqrestore_hw(&i8259A_lock, flags); } void enable_8259A_irq(unsigned int irq) @@ -106,13 +107,14 @@ void enable_8259A_irq(unsigned int irq) unsigned int mask = ~(1 << irq); unsigned long flags; - spin_lock_irqsave(&i8259A_lock, flags); + spin_lock_irqsave_hw(&i8259A_lock, flags); cached_irq_mask &= mask; if (irq & 8) outb(cached_slave_mask, PIC_SLAVE_IMR); else outb(cached_master_mask, PIC_MASTER_IMR); - spin_unlock_irqrestore(&i8259A_lock, flags); + pic_irq_unlock(irq); + spin_unlock_irqrestore_hw(&i8259A_lock, flags); } int i8259A_irq_pending(unsigned int irq) @@ -121,12 +123,12 @@ int i8259A_irq_pending(unsigned int irq) unsigned long flags; int ret; - spin_lock_irqsave(&i8259A_lock, flags); + spin_lock_irqsave_hw(&i8259A_lock, flags); if (irq < 8) ret = inb(PIC_MASTER_CMD) & mask; else ret = inb(PIC_SLAVE_CMD) & (mask >> 8); - spin_unlock_irqrestore(&i8259A_lock, flags); + spin_unlock_irqrestore_hw(&i8259A_lock, flags); return ret; } @@ -173,7 +175,7 @@ static void mask_and_ack_8259A(unsigned unsigned int irqmask = 1 << irq; unsigned long flags; - spin_lock_irqsave(&i8259A_lock, flags); + spin_lock_irqsave_hw(&i8259A_lock, flags); /* * Lightweight spurious IRQ detection. We do not want * to overdo spurious IRQ handling - it's usually a sign @@ -191,6 +193,15 @@ static void mask_and_ack_8259A(unsigned */ if (cached_irq_mask & irqmask) goto spurious_8259A_irq; +#ifdef CONFIG_ADEOS_CORE + if (irq == 0) { + /* Fast timer ack -- don't mask + (unless supposedly spurious) */ + outb(0x20,PIC_MASTER_CMD); + spin_unlock_irqrestore_hw(&i8259A_lock,flags); + return; + } +#endif /* CONFIG_ADEOS_CORE */ cached_irq_mask |= irqmask; handle_real_irq: @@ -204,7 +215,7 @@ handle_real_irq: outb(cached_master_mask, PIC_MASTER_IMR); outb(0x60+irq,PIC_MASTER_CMD); /* 'Specific EOI to master */ } - spin_unlock_irqrestore(&i8259A_lock, flags); + spin_unlock_irqrestore_hw(&i8259A_lock, flags); return; spurious_8259A_irq: @@ -305,7 +316,7 @@ void init_8259A(int auto_eoi) { unsigned long flags; - spin_lock_irqsave(&i8259A_lock, flags); + spin_lock_irqsave_hw(&i8259A_lock, flags); outb(0xff, PIC_MASTER_IMR); /* mask all of 8259A-1 */ outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-2 */ @@ -339,7 +350,7 @@ void init_8259A(int auto_eoi) outb(cached_master_mask, PIC_MASTER_IMR); /* restore master IRQ mask */ outb(cached_slave_mask, PIC_SLAVE_IMR); /* restore slave IRQ mask */ - spin_unlock_irqrestore(&i8259A_lock, flags); + spin_unlock_irqrestore_hw(&i8259A_lock, flags); } /* diff -uNrp linux-2.6.13/arch/i386/kernel/io_apic.c linux-2.6.13-adeos/arch/i386/kernel/io_apic.c --- linux-2.6.13/arch/i386/kernel/io_apic.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/io_apic.c 2005-09-17 19:46:23.000000000 +0200 @@ -175,18 +175,20 @@ static void mask_IO_APIC_irq (unsigned i { unsigned long flags; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); + pic_irq_lock(irq); __mask_IO_APIC_irq(irq); - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); } static void unmask_IO_APIC_irq (unsigned int irq) { unsigned long flags; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); __unmask_IO_APIC_irq(irq); - spin_unlock_irqrestore(&ioapic_lock, flags); + pic_irq_unlock(irq); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); } static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin) @@ -195,10 +197,10 @@ static void clear_IO_APIC_pin(unsigned i unsigned long flags; /* Check delivery_mode to be sure we're not clearing an SMI pin */ - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); *(((int*)&entry) + 0) = io_apic_read(apic, 0x10 + 2 * pin); *(((int*)&entry) + 1) = io_apic_read(apic, 0x11 + 2 * pin); - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); if (entry.delivery_mode == dest_SMI) return; @@ -207,10 +209,10 @@ static void clear_IO_APIC_pin(unsigned i */ memset(&entry, 0, sizeof(entry)); entry.mask = 1; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); io_apic_write(apic, 0x10 + 2 * pin, *(((int *)&entry) + 0)); io_apic_write(apic, 0x11 + 2 * pin, *(((int *)&entry) + 1)); - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); } static void clear_IO_APIC (void) @@ -232,7 +234,7 @@ static void set_ioapic_affinity_irq(unsi apicid_value = cpu_mask_to_apicid(cpumask); /* Prepare to do the io_apic_write */ apicid_value = apicid_value << 24; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); for (;;) { pin = entry->pin; if (pin == -1) @@ -242,7 +244,7 @@ static void set_ioapic_affinity_irq(unsi break; entry = irq_2_pin + entry->next; } - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); } #if defined(CONFIG_IRQBALANCE) @@ -1845,14 +1847,15 @@ static unsigned int startup_edge_ioapic_ int was_pending = 0; unsigned long flags; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); if (irq < 16) { disable_8259A_irq(irq); if (i8259A_irq_pending(irq)) was_pending = 1; } __unmask_IO_APIC_irq(irq); - spin_unlock_irqrestore(&ioapic_lock, flags); + pic_irq_unlock(irq); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); return was_pending; } @@ -1862,6 +1865,32 @@ static unsigned int startup_edge_ioapic_ * interrupt for real. This prevents IRQ storms from unhandled * devices. */ + +#ifdef CONFIG_ADEOS_CORE + +static void ack_edge_ioapic_irq (unsigned irq) + +{ + if ((irq_desc[irq].status & (IRQ_PENDING | IRQ_DISABLED)) == (IRQ_PENDING | IRQ_DISABLED)) + { + unsigned long flags; + spin_lock_irqsave_hw(&ioapic_lock,flags); + __mask_IO_APIC_irq(irq); + spin_unlock_irqrestore_hw(&ioapic_lock,flags); + } + + __ack_APIC_irq(); +} + +#ifdef CONFIG_IRQBALANCE +static void end_edge_ioapic_irq (unsigned irq) +{ + move_irq(irq); +} +#endif /* CONFIG_IRQBALANCE */ + +#else /* !CONFIG_ADEOS_CORE */ + static void ack_edge_ioapic_irq(unsigned int irq) { move_irq(irq); @@ -1871,6 +1900,8 @@ static void ack_edge_ioapic_irq(unsigned ack_APIC_irq(); } +#endif /* CONFIG_ADEOS_CORE */ + /* * Level triggered interrupts can just be masked, * and shutting down and starting up the interrupt @@ -1892,6 +1923,87 @@ static unsigned int startup_level_ioapic return 0; /* don't check for pending */ } +#ifdef CONFIG_ADEOS_CORE + +/* The standard Linux implementation for acknowledging IO-APIC + interrupts has been changed in order to guarantee that low priority + IRQs won't be delayed waiting for a high priority interrupt handler + to call end_level_ioapic_irq(). Therefore we immediately ack in + mask_and_ack_level_ioapic_irq(), still handling the 82093AA bugous + edge case. */ + +static unsigned long bugous_edge_triggers; + +static void end_level_ioapic_irq (unsigned irq) + +{ + unsigned long flags; + + move_irq(irq); + + spin_lock_irqsave_hw(&ioapic_lock,flags); + + if (test_and_clear_bit(irq,&bugous_edge_triggers)) + { + atomic_inc(&irq_mis_count); + __unmask_and_level_IO_APIC_irq(irq); + } + else + __unmask_IO_APIC_irq(irq); + + pic_irq_unlock(irq); + + spin_unlock_irqrestore_hw(&ioapic_lock,flags); +} + +static void mask_and_ack_level_ioapic_irq (unsigned irq) + +{ + unsigned long flags, v; + int i; + + i = IO_APIC_VECTOR(irq); + v = apic_read(APIC_TMR + ((i & ~0x1f) >> 1)); + + spin_lock_irqsave_hw(&ioapic_lock,flags); + + if (!(v & (1 << (i & 0x1f)))) + { + set_bit(irq,&bugous_edge_triggers); + __mask_and_edge_IO_APIC_irq(irq); + } + else + __mask_IO_APIC_irq(irq); + + spin_unlock_irqrestore_hw(&ioapic_lock,flags); + + __ack_APIC_irq(); +} + +#ifdef CONFIG_PCI_MSI + +#ifdef CONFIG_IRQBALANCE +static inline void end_edge_ioapic_vector(unsigned int vector) + +{ + int irq = vector_to_irq(vector); + + end_edge_ioapic_irq(irq); +} +#endif /* CONFIG_IRQBALANCE */ + +static inline void mask_and_ack_level_ioapic_vector(unsigned int vector) + +{ + int irq = vector_to_irq(vector); + + mask_and_ack_level_ioapic_irq(irq); +} + +#endif /* CONFIG_PCI_MSI */ + +#else /* !CONFIG_ADEOS_CORE */ + static void end_level_ioapic_irq (unsigned int irq) { unsigned long v; @@ -1932,6 +2044,8 @@ static void end_level_ioapic_irq (unsign } } +#endif /* CONFIG_ADEOS_CORE */ + #ifdef CONFIG_PCI_MSI static unsigned int startup_edge_ioapic_vector(unsigned int vector) { @@ -2069,7 +2183,7 @@ static void disable_lapic_irq (unsigned static void ack_lapic_irq (unsigned int irq) { - ack_APIC_irq(); + __ack_APIC_irq(); } static void end_lapic_irq (unsigned int i) { /* nothing */ } @@ -2340,12 +2454,12 @@ static int ioapic_suspend(struct sys_dev data = container_of(dev, struct sysfs_ioapic_data, dev); entry = data->entry; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); for (i = 0; i < nr_ioapic_registers[dev->id]; i ++, entry ++ ) { *(((int *)entry) + 1) = io_apic_read(dev->id, 0x11 + 2 * i); *(((int *)entry) + 0) = io_apic_read(dev->id, 0x10 + 2 * i); } - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); return 0; } @@ -2361,7 +2475,7 @@ static int ioapic_resume(struct sys_devi data = container_of(dev, struct sysfs_ioapic_data, dev); entry = data->entry; - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); reg_00.raw = io_apic_read(dev->id, 0); if (reg_00.bits.ID != mp_ioapics[dev->id].mpc_apicid) { reg_00.bits.ID = mp_ioapics[dev->id].mpc_apicid; @@ -2371,7 +2485,7 @@ static int ioapic_resume(struct sys_devi io_apic_write(dev->id, 0x11+2*i, *(((int *)entry)+1)); io_apic_write(dev->id, 0x10+2*i, *(((int *)entry)+0)); } - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); return 0; } @@ -2566,12 +2680,38 @@ int io_apic_set_pci_routing (int ioapic, if (!ioapic && (irq < 16)) disable_8259A_irq(irq); - spin_lock_irqsave(&ioapic_lock, flags); + spin_lock_irqsave_hw(&ioapic_lock, flags); io_apic_write(ioapic, 0x11+2*pin, *(((int *)&entry)+1)); io_apic_write(ioapic, 0x10+2*pin, *(((int *)&entry)+0)); - spin_unlock_irqrestore(&ioapic_lock, flags); + spin_unlock_irqrestore_hw(&ioapic_lock, flags); return 0; } #endif /*CONFIG_ACPI_BOOT*/ + +#if defined(CONFIG_ADEOS_CORE) && defined(CONFIG_SMP) + +cpumask_t __adeos_set_irq_affinity (unsigned irq, cpumask_t cpumask) + +{ + cpumask_t oldmask = irq_affinity[irq]; + + if (irq_desc[irq].handler->set_affinity == NULL) + return CPU_MASK_NONE; + + if (cpus_empty(cpumask)) + return oldmask; /* Return mask value -- no change. */ + + cpus_and(cpumask,cpumask,cpu_online_map); + + if (cpus_empty(cpumask) || irq_desc[irq].handler->set_affinity == NULL) + return CPU_MASK_NONE; /* Error -- bad mask value or non-routable IRQ. */ + + irq_affinity[irq] = cpumask; + irq_desc[irq].handler->set_affinity(irq,cpumask); + + return oldmask; +} + +#endif /* CONFIG_ADEOS_CORE && CONFIG_SMP */ diff -uNrp linux-2.6.13/arch/i386/kernel/nmi.c linux-2.6.13-adeos/arch/i386/kernel/nmi.c --- linux-2.6.13/arch/i386/kernel/nmi.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/nmi.c 2005-10-02 22:02:09.000000000 +0200 @@ -39,6 +39,7 @@ static unsigned int nmi_hz = HZ; static unsigned int nmi_perfctr_msr; /* the MSR to reset in NMI handler */ static unsigned int nmi_p4_cccr_val; extern void show_registers(struct pt_regs *regs); +static void default_nmi_watchdog_tick (struct pt_regs * regs); /* * lapic_nmi_owner tracks the ownership of the lapic NMI hardware: @@ -101,6 +102,11 @@ int nmi_active; (P4_CCCR_OVF_PMI0|P4_CCCR_THRESHOLD(15)|P4_CCCR_COMPLEMENT| \ P4_CCCR_COMPARE|P4_CCCR_REQUIRED|P4_CCCR_ESCR_SELECT(4)|P4_CCCR_ENABLE) +static void delay_10_ticks(void *ignored) +{ + mdelay((10*1000)/nmi_hz); // wait 10 ticks +} + static int __init check_nmi_watchdog(void) { unsigned int prev_nmi_count[NR_CPUS]; @@ -114,7 +120,11 @@ static int __init check_nmi_watchdog(voi for (cpu = 0; cpu < NR_CPUS; cpu++) prev_nmi_count[cpu] = per_cpu(irq_stat, cpu).__nmi_count; local_irq_enable(); - mdelay((10*1000)/nmi_hz); // wait 10 ticks +#ifdef CONFIG_SMP + smp_call_function(delay_10_ticks, NULL, 0, 1); +#else + delay_10_ticks(NULL); +#endif for (cpu = 0; cpu < NR_CPUS; cpu++) { #ifdef CONFIG_SMP @@ -150,6 +160,7 @@ static int __init setup_nmi_watchdog(cha if (nmi >= NMI_INVALID) return 0; + nmi_watchdog_tick = default_nmi_watchdog_tick; if (nmi == NMI_NONE) nmi_watchdog = nmi; /* @@ -480,9 +491,7 @@ void touch_nmi_watchdog (void) alert_counter[i] = 0; } -extern void die_nmi(struct pt_regs *, const char *msg); - -void nmi_watchdog_tick (struct pt_regs * regs) +static void default_nmi_watchdog_tick (struct pt_regs * regs) { /* @@ -577,3 +586,4 @@ EXPORT_SYMBOL(reserve_lapic_nmi); EXPORT_SYMBOL(release_lapic_nmi); EXPORT_SYMBOL(disable_timer_nmi_watchdog); EXPORT_SYMBOL(enable_timer_nmi_watchdog); +EXPORT_SYMBOL(touch_nmi_watchdog); diff -uNrp linux-2.6.13/arch/i386/kernel/process.c linux-2.6.13-adeos/arch/i386/kernel/process.c --- linux-2.6.13/arch/i386/kernel/process.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/process.c 2005-09-04 12:38:02.000000000 +0200 @@ -201,6 +201,9 @@ void cpu_idle(void) play_dead(); __get_cpu_var(irq_stat).idle_timestamp = jiffies; +#ifdef CONFIG_ADEOS_CORE + adeos_suspend_domain(); +#endif /* CONFIG_ADEOS_CORE */ idle(); } schedule(); diff -uNrp linux-2.6.13/arch/i386/kernel/smp.c linux-2.6.13-adeos/arch/i386/kernel/smp.c --- linux-2.6.13/arch/i386/kernel/smp.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/smp.c 2005-09-04 12:38:02.000000000 +0200 @@ -134,6 +134,11 @@ void __send_IPI_shortcut(unsigned int sh */ unsigned int cfg; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* !CONFIG_ADEOS_CORE */ + /* * Wait for idle. */ @@ -148,6 +153,10 @@ void __send_IPI_shortcut(unsigned int sh * Send the IPI. The write to APIC_ICR fires this off. */ apic_write_around(APIC_ICR, cfg); + +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* !CONFIG_ADEOS_CORE */ } void fastcall send_IPI_self(int vector) @@ -164,7 +173,11 @@ void send_IPI_mask_bitmask(cpumask_t cpu unsigned long cfg; unsigned long flags; +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_save(flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ WARN_ON(mask & ~cpus_addr(cpu_online_map)[0]); /* * Wait for idle. @@ -187,7 +200,11 @@ void send_IPI_mask_bitmask(cpumask_t cpu */ apic_write_around(APIC_ICR, cfg); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } void send_IPI_mask_sequence(cpumask_t mask, int vector) @@ -201,7 +218,11 @@ void send_IPI_mask_sequence(cpumask_t ma * should be modified to do 1 message per cluster ID - mbligh */ +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_save(flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ for (query_cpu = 0; query_cpu < NR_CPUS; ++query_cpu) { if (cpu_isset(query_cpu, mask)) { @@ -228,7 +249,11 @@ void send_IPI_mask_sequence(cpumask_t ma apic_write_around(APIC_ICR, cfg); } } +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#else /* !CONFIG_ADEOS_CORE */ local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } #include /* must come after the send_IPI functions above for inlining */ @@ -313,6 +338,10 @@ static inline void leave_mm (unsigned lo fastcall void smp_invalidate_interrupt(struct pt_regs *regs) { unsigned long cpu; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ cpu = get_cpu(); @@ -342,6 +371,9 @@ fastcall void smp_invalidate_interrupt(s smp_mb__after_clear_bit(); out: put_cpu_no_resched(); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } static void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm, @@ -402,6 +434,10 @@ void flush_tlb_current_task(void) { struct mm_struct *mm = current->mm; cpumask_t cpu_mask; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ preempt_disable(); cpu_mask = mm->cpu_vm_mask; @@ -410,6 +446,9 @@ void flush_tlb_current_task(void) local_flush_tlb(); if (!cpus_empty(cpu_mask)) flush_tlb_others(cpu_mask, mm, FLUSH_ALL); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ preempt_enable(); } @@ -437,6 +476,10 @@ void flush_tlb_page(struct vm_area_struc { struct mm_struct *mm = vma->vm_mm; cpumask_t cpu_mask; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ preempt_disable(); cpu_mask = mm->cpu_vm_mask; @@ -449,6 +492,10 @@ void flush_tlb_page(struct vm_area_struc leave_mm(smp_processor_id()); } +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ + if (!cpus_empty(cpu_mask)) flush_tlb_others(cpu_mask, mm, va); @@ -628,4 +675,3 @@ fastcall void smp_call_function_interrup atomic_inc(&call_data->finished); } } - diff -uNrp linux-2.6.13/arch/i386/kernel/smpboot.c linux-2.6.13-adeos/arch/i386/kernel/smpboot.c --- linux-2.6.13/arch/i386/kernel/smpboot.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/smpboot.c 2005-09-04 12:38:02.000000000 +0200 @@ -874,6 +874,10 @@ static int __devinit do_boot_cpu(int api ++cpucount; +#ifdef CONFIG_ADEOS_CORE + __adeos_apicid_2_cpuid[apicid] = cpu; +#endif /* CONFIG_ADEOS_CORE */ + /* * We can't use kernel_thread since we must avoid to * reschedule the child. @@ -1078,6 +1082,10 @@ static void __init smp_boot_cpus(unsigne int apicid, cpu, bit, kicked; unsigned long bogosum = 0; +#ifdef CONFIG_ADEOS_CORE + __adeos_apicid_2_cpuid[adeos_smp_apic_id()] = 0; +#endif /* CONFIG_ADEOS_CORE */ + /* * Setup boot CPU information */ diff -uNrp linux-2.6.13/arch/i386/kernel/time.c linux-2.6.13-adeos/arch/i386/kernel/time.c --- linux-2.6.13/arch/i386/kernel/time.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/time.c 2005-09-04 12:38:02.000000000 +0200 @@ -263,11 +263,12 @@ static inline void do_timer_interrupt(in * This will also deassert NMI lines for the watchdog if run * on an 82489DX-based system. */ - spin_lock(&i8259A_lock); + unsigned long flags; + spin_lock_irqsave_hw_cond(&i8259A_lock,flags); outb(0x0c, PIC_MASTER_OCW3); /* Ack the IRQ; AEOI will end it automatically. */ inb(PIC_MASTER_POLL); - spin_unlock(&i8259A_lock); + spin_unlock_irqrestore_hw_cond(&i8259A_lock,flags); } #endif diff -uNrp linux-2.6.13/arch/i386/kernel/timers/timer_pit.c linux-2.6.13-adeos/arch/i386/kernel/timers/timer_pit.c --- linux-2.6.13/arch/i386/kernel/timers/timer_pit.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/timers/timer_pit.c 2005-09-04 12:38:02.000000000 +0200 @@ -99,6 +99,10 @@ static unsigned long get_offset_pit(void */ unsigned long jiffies_t; +#ifdef CONFIG_ADEOS_CORE + if (!__adeos_pipeline_head_p(adp_root)) + return 0; /* We don't really own the PIT. */ +#endif /* CONFIG_ADEOS_CORE */ spin_lock_irqsave(&i8253_lock, flags); /* timer count may underflow right here */ outb_p(0x00, PIT_MODE); /* latch the count ASAP */ diff -uNrp linux-2.6.13/arch/i386/kernel/timers/timer_tsc.c linux-2.6.13-adeos/arch/i386/kernel/timers/timer_tsc.c --- linux-2.6.13/arch/i386/kernel/timers/timer_tsc.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/timers/timer_tsc.c 2005-09-17 22:47:21.000000000 +0200 @@ -367,6 +367,20 @@ static void mark_offset_tsc(void) rdtsc(last_tsc_low, last_tsc_high); +#ifdef CONFIG_ADEOS_CORE + if (!__adeos_pipeline_head_p(adp_root)) { + /* If Linux does not actually own the timer, clock + ticks will be posted by some higher level domain to + us, and we expect it to do this right and never + lose any of them; so we just need to update the + monotonic base here. */ + this_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low; + monotonic_base += cycles_2_ns(this_offset - last_offset); + write_sequnlock(&monotonic_lock); + return; + } +#endif /* CONFIG_ADEOS_CORE */ + spin_lock(&i8253_lock); outb_p(0x00, PIT_MODE); /* latch the count ASAP */ diff -uNrp linux-2.6.13/arch/i386/kernel/traps.c linux-2.6.13-adeos/arch/i386/kernel/traps.c --- linux-2.6.13/arch/i386/kernel/traps.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/kernel/traps.c 2005-10-02 22:00:30.000000000 +0200 @@ -96,6 +96,9 @@ static int kstack_depth_to_print = 24; struct notifier_block *i386die_chain; static DEFINE_SPINLOCK(die_notifier_lock); +void (*nmi_watchdog_tick) (struct pt_regs * regs); +EXPORT_SYMBOL(nmi_watchdog_tick); + int register_die_notifier(struct notifier_block *nb) { int err = 0; @@ -143,6 +146,13 @@ void show_trace(struct task_struct *task { unsigned long ebp; +#ifdef CONFIG_ADEOS_CORE + if (!task && adp_current != adp_root) + { + printk("Not executing in the root domain, no trace available\n"); + return; + } +#endif /* CONFIG_ADEOS_CORE */ if (!task) task = current; @@ -228,6 +238,11 @@ void show_registers(struct pt_regs *regs regs->esi, regs->edi, regs->ebp, esp); printk("ds: %04x es: %04x ss: %04x\n", regs->xds & 0xffff, regs->xes & 0xffff, ss); +#ifdef CONFIG_ADEOS_CORE + if (adp_current != adp_root) + printk("Adeos domain %s",adp_current->name); + else +#endif /* CONFIG_ADEOS_CORE */ printk("Process %s (pid: %d, threadinfo=%p task=%p)", current->comm, current->pid, current_thread_info(), current); /* @@ -311,6 +326,10 @@ void die(const char * str, struct pt_reg }; static int die_counter; +#ifdef CONFIG_ADEOS_CORE + if (adp_current != adp_root) + adeos_set_printk_sync(adp_current); +#endif /* CONFIG_ADEOS_CORE */ if (die.lock_owner != raw_smp_processor_id()) { console_verbose(); spin_lock_irq(&die.lock); @@ -568,6 +587,9 @@ static DEFINE_SPINLOCK(nmi_print_lock); void die_nmi (struct pt_regs *regs, const char *msg) { +#ifdef CONFIG_ADEOS_CORE + adeos_set_printk_sync(adp_current); +#endif /* CONFIG_ADEOS_CORE */ spin_lock(&nmi_print_lock); /* * We are in trouble anyway, lets at least try @@ -578,6 +600,9 @@ void die_nmi (struct pt_regs *regs, cons printk(" on CPU%d, eip %08lx, registers:\n", smp_processor_id(), regs->eip); show_registers(regs); +#ifdef CONFIG_ADEOS_CORE + __adeos_dump_state(); +#endif /* CONFIG_ADEOS_CORE */ printk("console shuts up ...\n"); console_silent(); spin_unlock(&nmi_print_lock); @@ -594,6 +619,8 @@ void die_nmi (struct pt_regs *regs, cons do_exit(SIGSEGV); } +EXPORT_SYMBOL(die_nmi); + static void default_do_nmi(struct pt_regs * regs) { unsigned char reason = 0; @@ -643,6 +670,9 @@ fastcall void do_nmi(struct pt_regs * re { int cpu; +#ifdef CONFIG_ADEOS_CORE + if (adp_current == adp_root) +#endif /* CONFIG_ADEOS_CORE */ nmi_enter(); cpu = smp_processor_id(); @@ -659,6 +689,9 @@ fastcall void do_nmi(struct pt_regs * re if (!nmi_callback(regs, cpu)) default_do_nmi(regs); +#ifdef CONFIG_ADEOS_CORE + if (adp_current == adp_root) +#endif /* CONFIG_ADEOS_CORE */ nmi_exit(); } @@ -978,12 +1011,19 @@ asmlinkage void math_state_restore(struc { struct thread_info *thread = current_thread_info(); struct task_struct *tsk = thread->task; +#ifdef CONFIG_ADEOS_CORE + unsigned long flags; + adeos_hw_local_irq_save(flags); +#endif /* CONFIG_ADEOS_CORE */ clts(); /* Allow maths ops (or we recurse) */ if (!tsk_used_math(tsk)) init_fpu(tsk); restore_fpu(tsk); thread->status |= TS_USEDFPU; /* So we fnsave on switch_to() */ +#ifdef CONFIG_ADEOS_CORE + adeos_hw_local_irq_restore(flags); +#endif /* CONFIG_ADEOS_CORE */ } #ifndef CONFIG_MATH_EMULATION diff -uNrp linux-2.6.13/arch/i386/mm/fault.c linux-2.6.13-adeos/arch/i386/mm/fault.c --- linux-2.6.13/arch/i386/mm/fault.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/mm/fault.c 2005-09-04 12:38:02.000000000 +0200 @@ -224,12 +224,16 @@ fastcall void do_page_fault(struct pt_re /* get the address */ __asm__("movl %%cr2,%0":"=r" (address)); +#ifdef CONFIG_ADEOS_CORE + adeos_hw_sti(); +#endif /* CONFIG_ADEOS_CORE */ + if (notify_die(DIE_PAGE_FAULT, "page fault", regs, error_code, 14, SIGSEGV) == NOTIFY_STOP) return; /* It's safe to allow irq's after cr2 has been saved */ if (regs->eflags & (X86_EFLAGS_IF|VM_MASK)) - local_irq_enable(); + local_irq_enable(); tsk = current; diff -uNrp linux-2.6.13/arch/i386/mm/ioremap.c linux-2.6.13-adeos/arch/i386/mm/ioremap.c --- linux-2.6.13/arch/i386/mm/ioremap.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/arch/i386/mm/ioremap.c 2005-09-04 12:38:02.000000000 +0200 @@ -17,6 +17,9 @@ #include #include #include +#ifdef CONFIG_ADEOS_CORE +#include +#endif /* CONFIG_ADEOS_CORE */ #define ISA_START_ADDRESS 0xa0000 #define ISA_END_ADDRESS 0x100000 @@ -93,6 +96,9 @@ static int ioremap_page_range(unsigned l err = ioremap_pud_range(pgd, addr, next, phys_addr+addr, flags); if (err) break; +#ifdef CONFIG_ADEOS_CORE + set_pgdir(addr, *pgd); +#endif /* CONFIG_ADEOS_CORE */ } while (pgd++, addr = next, addr != end); spin_unlock(&init_mm.page_table_lock); flush_tlb_all(); diff -uNrp linux-2.6.13/include/asm-i386/adeos.h linux-2.6.13-adeos/include/asm-i386/adeos.h --- linux-2.6.13/include/asm-i386/adeos.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/include/asm-i386/adeos.h 2005-10-01 22:04:45.000000000 +0200 @@ -0,0 +1,511 @@ +/* + * include/asm-i386/adeos.h + * + * Copyright (C) 2002 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __I386_ADEOS_H +#define __I386_ADEOS_H + +#include +#include +#include +#include +#include + +#define ADEOS_ARCH_STRING "r13/x86" +#define ADEOS_MAJOR_NUMBER 13 +#define ADEOS_MINOR_NUMBER 255 + +extern int adp_pipelined; + +#ifdef CONFIG_SMP + +#include +#include +#include + +#define ADEOS_NR_CPUS NR_CPUS +#define ADEOS_CRITICAL_VECTOR 0xf9 /* Used by adeos_critical_enter/exit() */ +#define ADEOS_CRITICAL_IPI (ADEOS_CRITICAL_VECTOR - FIRST_EXTERNAL_VECTOR) + +static __inline int adeos_smp_apic_id(void) { + return GET_APIC_ID(*(unsigned long *)(APIC_BASE+APIC_ID)); +} + +extern volatile u8 __adeos_apicid_2_cpuid[]; + +int __adeos_hw_cpuid(void); + +cpumask_t __adeos_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +#define adeos_processor_id() __adeos_hw_cpuid() + +#define adeos_declare_cpuid int cpuid +#define adeos_load_cpuid() do { \ + (cpuid) = adeos_processor_id(); \ + } while(0) +#define adeos_lock_cpu(flags) do { \ + adeos_hw_local_irq_save(flags); \ + (cpuid) = adeos_processor_id(); \ + } while(0) +#define adeos_unlock_cpu(flags) adeos_hw_local_irq_restore(flags) +#define adeos_get_cpu(flags) adeos_lock_cpu(flags) +#define adeos_put_cpu(flags) adeos_unlock_cpu(flags) +#define adp_current (adp_cpu_current[adeos_processor_id()]) + +#else /* !CONFIG_SMP */ + +#define ADEOS_NR_CPUS 1 +#define adeos_processor_id() 0 +/* Array references using this index should be optimized out. */ +#define adeos_declare_cpuid const int cpuid = 0 +#define adeos_load_cpuid() /* nop */ +#define adeos_lock_cpu(flags) adeos_hw_local_irq_save(flags) +#define adeos_unlock_cpu(flags) adeos_hw_local_irq_restore(flags) +#define adeos_get_cpu(flags) do { flags = 0; } while(0) +#define adeos_put_cpu(flags) /* nop */ +#define adp_current (adp_cpu_current[0]) + +#endif /* CONFIG_SMP */ + + /* IDT fault vectors */ +#define ADEOS_NR_FAULTS 32 +/* Pseudo-vectors used for kernel events */ +#define ADEOS_FIRST_KEVENT ADEOS_NR_FAULTS +#define ADEOS_SYSCALL_PROLOGUE (ADEOS_FIRST_KEVENT) +#define ADEOS_SYSCALL_EPILOGUE (ADEOS_FIRST_KEVENT + 1) +#define ADEOS_SCHEDULE_HEAD (ADEOS_FIRST_KEVENT + 2) +#define ADEOS_SCHEDULE_TAIL (ADEOS_FIRST_KEVENT + 3) +#define ADEOS_ENTER_PROCESS (ADEOS_FIRST_KEVENT + 4) +#define ADEOS_EXIT_PROCESS (ADEOS_FIRST_KEVENT + 5) +#define ADEOS_SIGNAL_PROCESS (ADEOS_FIRST_KEVENT + 6) +#define ADEOS_KICK_PROCESS (ADEOS_FIRST_KEVENT + 7) +#define ADEOS_RENICE_PROCESS (ADEOS_FIRST_KEVENT + 8) +#define ADEOS_USER_EVENT (ADEOS_FIRST_KEVENT + 9) +#define ADEOS_LAST_KEVENT (ADEOS_USER_EVENT) + +#define ADEOS_NR_EVENTS (ADEOS_LAST_KEVENT + 1) + +typedef struct adevinfo { + + unsigned domid; + unsigned event; + void *evdata; + + volatile int propagate; /* Private */ + +} adevinfo_t; + +typedef struct adsysinfo { + + int ncpus; /* Number of CPUs on board */ + + u64 cpufreq; /* CPU frequency (in Hz) */ + + /* Arch-dependent block */ + + struct { + unsigned tmirq; /* Timer tick IRQ */ + u64 tmfreq; /* Timer frequency */ + } archdep; + +} adsysinfo_t; + +#ifdef CONFIG_X86_LOCAL_APIC +/* We must cover the whole IRQ space to map the local timer interrupt + (#207). */ +#ifdef CONFIG_PCI_MSI +#define IPIPE_NR_XIRQS NR_IRQS +#else /* CONFIG_PCI_MSI */ +#define IPIPE_NR_XIRQS 224 +#endif /* CONFIG_PCI_MSI */ +/* If the APIC is enabled, then we expose four service vectors in the + APIC space which are freely available to domains. */ +#define ADEOS_SERVICE_VECTOR0 0xf5 +#define ADEOS_SERVICE_IPI0 (ADEOS_SERVICE_VECTOR0 - FIRST_EXTERNAL_VECTOR) +#define ADEOS_SERVICE_VECTOR1 0xf6 +#define ADEOS_SERVICE_IPI1 (ADEOS_SERVICE_VECTOR1 - FIRST_EXTERNAL_VECTOR) +#define ADEOS_SERVICE_VECTOR2 0xf7 +#define ADEOS_SERVICE_IPI2 (ADEOS_SERVICE_VECTOR2 - FIRST_EXTERNAL_VECTOR) +#define ADEOS_SERVICE_VECTOR3 0xf8 +#define ADEOS_SERVICE_IPI3 (ADEOS_SERVICE_VECTOR3 - FIRST_EXTERNAL_VECTOR) +#else /* !CONFIG_X86_LOCAL_APIC */ +#define IPIPE_NR_XIRQS NR_IRQS +#endif /* CONFIG_X86_LOCAL_APIC */ + +/* Number of virtual IRQs */ +#define IPIPE_NR_VIRQS BITS_PER_LONG +/* First virtual IRQ # */ +#define IPIPE_VIRQ_BASE (((IPIPE_NR_XIRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) * BITS_PER_LONG) +/* Total number of IRQ slots */ +#define IPIPE_NR_IRQS (IPIPE_VIRQ_BASE + IPIPE_NR_VIRQS) +/* Number of indirect words needed to map the whole IRQ space. */ +#define IPIPE_IRQ_IWORDS ((IPIPE_NR_IRQS + BITS_PER_LONG - 1) / BITS_PER_LONG) +#define IPIPE_IRQ_IMASK (BITS_PER_LONG - 1) +#define IPIPE_IRQ_ISHIFT 5 /* 2^5 for 32bits arch. */ + +#define IPIPE_IRQMASK_ANY (~0L) +#define IPIPE_IRQMASK_VIRT (IPIPE_IRQMASK_ANY << (IPIPE_VIRQ_BASE / BITS_PER_LONG)) + +typedef struct adomain { + + /* -- Section: offset-based references are made on these fields + from inline assembly code. Please don't move or reorder. */ + void (*dswitch)(void); /* Domain switch hook */ +#ifdef CONFIG_ADEOS_THREADS + int *esp[ADEOS_NR_CPUS]; /* Domain stack pointers */ +#endif /* CONFIG_ADEOS_THREADS */ + /* -- End of section. */ + + struct list_head p_link; /* Link in pipeline */ + + struct adcpudata { + volatile unsigned long status; + volatile unsigned long irq_pending_hi; + volatile unsigned long irq_pending_lo[IPIPE_IRQ_IWORDS]; + volatile unsigned irq_hits[IPIPE_NR_IRQS]; +#ifdef CONFIG_ADEOS_THREADS + adevinfo_t event_info; +#endif /* CONFIG_ADEOS_THREADS */ + } cpudata[ADEOS_NR_CPUS]; + + struct { + int (*acknowledge)(unsigned irq); + void (*handler)(unsigned irq); + unsigned long control; + } irqs[IPIPE_NR_IRQS]; + + struct { + void (*handler)(adevinfo_t *evinfo); + } events[ADEOS_NR_EVENTS]; + + struct adomain *m_link; /* Link in mutex sleep queue */ + + unsigned long flags; + + unsigned domid; + + const char *name; + + int priority; + + int ptd_keymax; + int ptd_keycount; + unsigned long ptd_keymap; + void (*ptd_setfun)(int, void *); + void *(*ptd_getfun)(int); + +#ifdef CONFIG_ADEOS_THREADS + int *estackbase[ADEOS_NR_CPUS]; +#endif /* CONFIG_ADEOS_THREADS */ + +} adomain_t; + +/* The following macros must be used hw interrupts off. */ + +#define __adeos_set_irq_bit(adp,cpuid,irq) \ +do { \ + if (!test_bit(IPIPE_LOCK_FLAG,&(adp)->irqs[irq].control)) { \ + __set_bit(irq & IPIPE_IRQ_IMASK,&(adp)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + __set_bit(irq >> IPIPE_IRQ_ISHIFT,&(adp)->cpudata[cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __adeos_clear_pend(adp,cpuid,irq) \ +do { \ + __clear_bit(irq & IPIPE_IRQ_IMASK,&(adp)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + if ((adp)->cpudata[cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT] == 0) \ + __clear_bit(irq >> IPIPE_IRQ_ISHIFT,&(adp)->cpudata[cpuid].irq_pending_hi); \ +} while(0) + +#define __adeos_lock_irq(adp,cpuid,irq) \ +do { \ + if (!test_and_set_bit(IPIPE_LOCK_FLAG,&(adp)->irqs[irq].control)) \ + __adeos_clear_pend(adp,cpuid,irq); \ +} while(0) + +#define __adeos_unlock_irq(adp,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + if (test_and_clear_bit(IPIPE_LOCK_FLAG,&(adp)->irqs[irq].control)) \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) \ + if ((adp)->cpudata[__cpuid].irq_hits[irq] > 0) { /* We need atomic ops next. */ \ + set_bit(irq & IPIPE_IRQ_IMASK,&(adp)->cpudata[__cpuid].irq_pending_lo[irq >> IPIPE_IRQ_ISHIFT]); \ + set_bit(irq >> IPIPE_IRQ_ISHIFT,&(adp)->cpudata[__cpuid].irq_pending_hi); \ + } \ +} while(0) + +#define __adeos_clear_irq(adp,irq) \ +do { \ + int __cpuid, __nr_cpus = num_online_cpus(); \ + clear_bit(IPIPE_LOCK_FLAG,&(adp)->irqs[irq].control); \ + for (__cpuid = 0; __cpuid < __nr_cpus; __cpuid++) { \ + (adp)->cpudata[__cpuid].irq_hits[irq] = 0; \ + __adeos_clear_pend(adp,__cpuid,irq); \ + } \ +} while(0) + +#define adeos_virtual_irq_p(irq) ((irq) >= IPIPE_VIRQ_BASE && \ + (irq) < IPIPE_NR_IRQS) + +#define adeos_hw_save_flags_and_sti(x) __asm__ __volatile__("pushfl ; popl %0 ; sti":"=g" (x): /* no input */ :"memory") +#define adeos_hw_cli() __asm__ __volatile__("cli": : :"memory") +#define adeos_hw_sti() __asm__ __volatile__("sti": : :"memory") +#define adeos_hw_local_irq_save(x) __asm__ __volatile__("pushfl ; popl %0 ; cli":"=g" (x): /* no input */ :"memory") +#define adeos_hw_local_irq_restore(x) __asm__ __volatile__("pushl %0 ; popfl": /* no output */ :"g" (x):"memory", "cc") +#define adeos_hw_local_irq_flags(x) __asm__ __volatile__("pushfl ; popl %0":"=g" (x): /* no input */) +#define adeos_hw_test_iflag(x) ((x) & (1<<9)) +#define adeos_hw_irqs_disabled() \ +({ \ + unsigned long flags; \ + adeos_hw_local_irq_flags(flags); \ + !adeos_hw_test_iflag(flags); \ +}) + +#define adeos_hw_tsc(t) __asm__ __volatile__("rdtsc" : "=A" (t)) +#define adeos_cpu_freq() ({ unsigned long long __freq = cpu_has_tsc?(1000LL * cpu_khz):CLOCK_TICK_RATE; __freq; }) + +#ifdef RAW_SPIN_LOCK_UNLOCKED +#define adeos_spin_lock(x) __raw_spin_lock(x) +#define adeos_spin_unlock(x) __raw_spin_unlock(x) +#define adeos_spin_trylock(x) __raw_spin_trylock(x) +#define adeos_write_lock(x) __raw_write_lock(x) +#define adeos_write_unlock(x) __raw_write_unlock(x) +#define adeos_write_trylock(x) __raw_write_trylock(x) +#define adeos_read_lock(x) __raw_read_lock(x) +#define adeos_read_unlock(x) __raw_read_unlock(x) +#else /* !RAW_SPIN_LOCK_UNLOCKED */ +#define adeos_spin_lock(x) _spin_lock(x) +#define adeos_spin_unlock(x) _spin_unlock(x) +#define adeos_spin_trylock(x) _spin_trylock(x) +#define adeos_write_lock(x) _write_lock(x) +#define adeos_write_unlock(x) _write_unlock(x) +#define adeos_write_trylock(x) _write_trylock(x) +#define adeos_read_lock(x) _read_lock(x) +#define adeos_read_unlock(x) _read_unlock(x) +#define raw_spinlock_t spinlock_t +#define RAW_SPIN_LOCK_UNLOCKED SPIN_LOCK_UNLOCKED +#define raw_rwlock_t rwlock_t +#define RAW_RW_LOCK_UNLOCKED RW_LOCK_UNLOCKED +#endif /* RAW_RW_LOCK_UNLOCKED */ + +#define spin_lock_irqsave_hw(lock,flags) adeos_spin_lock_irqsave(lock, flags) +#define spin_unlock_irqrestore_hw(lock,flags) adeos_spin_unlock_irqrestore(lock, flags) + +#define adeos_spin_lock_irqsave(x,flags) \ +do { \ + adeos_hw_local_irq_save(flags); \ + adeos_spin_lock(x); \ +} while (0) + +#define adeos_spin_unlock_irqrestore(x,flags) \ +do { \ + adeos_spin_unlock(x); \ + adeos_hw_local_irq_restore(flags); \ +} while (0) + +#define adeos_spin_lock_disable(x) \ +do { \ + adeos_hw_cli(); \ + adeos_spin_lock(x); \ +} while (0) + +#define adeos_spin_unlock_enable(x) \ +do { \ + adeos_spin_unlock(x); \ + adeos_hw_sti(); \ +} while (0) + +#define adeos_read_lock_irqsave(lock, flags) \ +do { \ + adeos_hw_local_irq_save(flags); \ + adeos_read_lock(lock); \ +} while (0) + +#define adeos_read_unlock_irqrestore(lock, flags) \ +do { \ + adeos_read_unlock(lock); \ + adeos_hw_local_irq_restore(flags); \ +} while (0) + +#define adeos_write_lock_irqsave(lock, flags) \ +do { \ + adeos_hw_local_irq_save(flags); \ + adeos_write_lock(lock); \ +} while (0) + +#define adeos_write_unlock_irqrestore(lock, flags) \ +do { \ + adeos_write_unlock(lock); \ + adeos_hw_local_irq_restore(flags); \ +} while (0) + +#ifndef STR +#define __STR(x) #x +#define STR(x) __STR(x) +#endif + +#ifndef SYMBOL_NAME_STR +#define SYMBOL_NAME_STR(X) #X +#endif + +/* Private interface -- Internal use only */ + +struct adattr; + +void __adeos_init(void); + +void __adeos_init_domain(adomain_t *adp, + struct adattr *attr); + +void __adeos_cleanup_domain(adomain_t *adp); + +void __adeos_send_IPI_mask(cpumask_t mask, + int vector); + +void __adeos_send_IPI_allbutself(int vector); + +#ifdef CONFIG_ADEOS_THREADS + +#ifdef CONFIG_SMP + +/* + * __adeos_switch_to() -- Switch domain contexts. The current domain + * ("out") is switched out while the domain pointed by "in" is + * switched in. The current cpu identifier which is always known from + * callers is also passed to save a few cycles. This code works out + * the following tasks: - build a resume frame for the suspended + * domain, - save it's stack pointer, - load the incoming domain's + * stack pointer, - update the global domain descriptor pointer, - + * then finally activate the resume frame of the incoming domain. + * + * SMP version also provides for safe CPU migration (i.e. the domain + * may be switched back in on behalf of a different CPU than the one + * which switched it out). + * + * NOTE: dswitch() takes no argument, so there is no specific handling + * of CONFIG_REGPARM needed. This routine must be called with hw + * interrupts off. + */ + +static inline void __adeos_switch_to (adomain_t *out, adomain_t *in, int cpuid) + +{ + __asm__ __volatile__( \ + "pushl %%ebp\n\t" \ + "pushl %%ecx\n\t" \ + "pushl %%ebx\n\t" \ + "pushl %%edi\n\t" \ + "pushl %%esi\n\t" \ + "movl %%eax, %%ecx\n\t" \ + "leal adp_cpu_current(,%%eax,4),%%eax\n\t" \ + "xchg (%%eax), %%edx\n\t" \ + "pushl %%edx\n\t" \ + "pushl $1f\n\t" \ + "sall $2,%%ecx\n\t" \ + "addl $4,%%ecx\n\t" \ + "movl %%esp, (%%ecx,%%edx)\n\t" \ + "movl (%%eax), %%eax\n\t" \ + "movl (%%ecx,%%eax), %%esp\n\t" \ + "ret\n\t" \ + /* Call domain switch hook (if any) */ +"1: popl %%eax\n\t" \ + "movl (%%eax), %%eax\n\t" \ + "testl %%eax,%%eax\n\t" \ + "je 2f\n\t" \ + "call *%%eax\n\t" \ + /* Domain resume point */ +"2: popl %%esi\n\t" \ + "popl %%edi\n\t" \ + "popl %%ebx\n\t" \ + "popl %%ecx\n\t" \ + "popl %%ebp\n\t" \ + : /* no output */ \ + : "a" (cpuid), "d" (in) \ + : "memory", "cc"); + + barrier(); +} + +#else /* !CONFIG_SMP */ + +static inline void __adeos_switch_to (adomain_t *out, adomain_t *in, int cpuid) + +{ + __asm__ __volatile__( \ + "pushl %%ebp\n\t" \ + "pushl %%edx\n\t" \ + "pushl %%ecx\n\t" \ + "pushl %%ebx\n\t" \ + "pushl %%edi\n\t" \ + "pushl %%esi\n\t" \ + "movl "SYMBOL_NAME_STR(adp_cpu_current)", %%edx\n\t" \ + "pushl %%edx\n\t" \ + "pushl $1f\n\t" \ + "movl %%esp, 4(%%edx)\n\t" \ + "movl 4(%%eax), %%esp\n\t" \ + "movl %%eax, "SYMBOL_NAME_STR(adp_cpu_current)"\n\t" \ + "ret\n\t" \ + /* Call domain switch hook (if any) */ +"1: popl %%eax\n\t" \ + "movl (%%eax),%%eax\n\t" \ + "testl %%eax,%%eax\n\t" \ + "je 2f\n\t" \ + "call *%%eax\n\t" \ + /* Domain resume point */ +"2: popl %%esi\n\t" \ + "popl %%edi\n\t" \ + "popl %%ebx\n\t" \ + "popl %%ecx\n\t" \ + "popl %%edx\n\t" \ + "popl %%ebp\n\t" \ + : /* no output */ \ + : "a" (in)); +} + +#endif /* CONFIG_SMP */ + +#endif /* CONFIG_ADEOS_THREADS */ + +#define __adeos_check_platform() do { } while(0) + +#define __adeos_init_platform() do { } while(0) + +void __adeos_enable_pipeline(void); + +void __adeos_disable_pipeline(void); + +void __adeos_init_stage(adomain_t *adp); + +void fastcall __adeos_sync_stage(unsigned long syncmask); + +int __adeos_ack_system_irq(unsigned irq); + +int __adeos_handle_irq(struct pt_regs regs); + +extern struct pt_regs __adeos_tick_regs[]; + +extern int __adeos_tick_irq; + +#ifdef CONFIG_X86_LOCAL_APIC +extern int __adeos_apic_mapped; +#endif /* CONFIG_X86_LOCAL_APIC */ + +#endif /* !__I386_ADEOS_H */ diff -uNrp linux-2.6.13/include/asm-i386/apic.h linux-2.6.13-adeos/include/asm-i386/apic.h --- linux-2.6.13/include/asm-i386/apic.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/apic.h 2005-10-02 22:00:56.000000000 +0200 @@ -82,7 +82,13 @@ int get_physical_broadcast(void); # define apic_write_around(x,y) apic_write_atomic((x),(y)) #endif +#ifdef CONFIG_ADEOS_CORE +#define ack_APIC_irq() do { if (!adp_pipelined) __ack_APIC_irq(); } while(0) +static inline void __ack_APIC_irq(void) +#else /* !CONFIG_ADEOS_CORE */ +#define __ack_APIC_irq() ack_APIC_irq() static inline void ack_APIC_irq(void) +#endif /* CONFIG_ADEOS_CORE */ { /* * ack_APIC_irq() actually gets compiled as a single instruction: @@ -117,7 +123,7 @@ extern int reserve_lapic_nmi(void); extern void release_lapic_nmi(void); extern void disable_timer_nmi_watchdog(void); extern void enable_timer_nmi_watchdog(void); -extern void nmi_watchdog_tick (struct pt_regs * regs); +extern void (*nmi_watchdog_tick) (struct pt_regs * regs); extern int APIC_init_uniprocessor (void); extern void disable_APIC_timer(void); extern void enable_APIC_timer(void); diff -uNrp linux-2.6.13/include/asm-i386/io_apic.h linux-2.6.13-adeos/include/asm-i386/io_apic.h --- linux-2.6.13/include/asm-i386/io_apic.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/io_apic.h 2005-09-16 17:53:26.000000000 +0200 @@ -16,8 +16,14 @@ #ifdef CONFIG_PCI_MSI static inline int use_pci_vector(void) {return 1;} static inline void disable_edge_ioapic_vector(unsigned int vector) { } +#ifdef CONFIG_ADEOS_CORE +#ifndef CONFIG_IRQBALANCE +static inline void end_edge_ioapic_vector (unsigned int vector) { } +#endif /* !CONFIG_IRQBALANCE */ +#else /* !CONFIG_ADEOS_CORE */ static inline void mask_and_ack_level_ioapic_vector(unsigned int vector) { } static inline void end_edge_ioapic_vector (unsigned int vector) { } +#endif /* CONFIG_ADEOS_CORE */ #define startup_level_ioapic startup_level_ioapic_vector #define shutdown_level_ioapic mask_IO_APIC_vector #define enable_level_ioapic unmask_IO_APIC_vector @@ -35,8 +41,14 @@ static inline void end_edge_ioapic_vecto #else static inline int use_pci_vector(void) {return 0;} static inline void disable_edge_ioapic_irq(unsigned int irq) { } +#ifdef CONFIG_ADEOS_CORE +#ifndef CONFIG_IRQBALANCE +static inline void end_edge_ioapic_irq (unsigned int irq) { } +#endif /* !CONFIG_IRQBALANCE */ +#else /* CONFIG_ADEOS_CORE */ static inline void mask_and_ack_level_ioapic_irq(unsigned int irq) { } static inline void end_edge_ioapic_irq (unsigned int irq) { } +#endif /* CONFIG_ADEOS_CORE */ #define startup_level_ioapic startup_level_ioapic_irq #define shutdown_level_ioapic mask_IO_APIC_irq #define enable_level_ioapic unmask_IO_APIC_irq diff -uNrp linux-2.6.13/include/asm-i386/mach-default/do_timer.h linux-2.6.13-adeos/include/asm-i386/mach-default/do_timer.h --- linux-2.6.13/include/asm-i386/mach-default/do_timer.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/mach-default/do_timer.h 2005-09-04 12:38:02.000000000 +0200 @@ -49,15 +49,16 @@ static inline void do_timer_interrupt_ho **/ static inline int do_timer_overflow(int count) { + unsigned long flags; int i; - spin_lock(&i8259A_lock); + spin_lock_irqsave_hw_cond(&i8259A_lock, flags); /* * This is tricky when I/O APICs are used; * see do_timer_interrupt(). */ i = inb(0x20); - spin_unlock(&i8259A_lock); + spin_unlock_irqrestore_hw_cond(&i8259A_lock, flags); /* assumption about timer being IRQ0 */ if (i & 0x01) { diff -uNrp linux-2.6.13/include/asm-i386/mach-visws/do_timer.h linux-2.6.13-adeos/include/asm-i386/mach-visws/do_timer.h --- linux-2.6.13/include/asm-i386/mach-visws/do_timer.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/mach-visws/do_timer.h 2005-09-04 12:38:02.000000000 +0200 @@ -28,15 +28,16 @@ static inline void do_timer_interrupt_ho static inline int do_timer_overflow(int count) { + unsigned long flags; int i; - spin_lock(&i8259A_lock); + spin_lock_irqsave_hw_cond(&i8259A_lock, flags); /* * This is tricky when I/O APICs are used; * see do_timer_interrupt(). */ i = inb(0x20); - spin_unlock(&i8259A_lock); + spin_unlock_irqrestore_hw_cond(&i8259A_lock, flags); /* assumption about timer being IRQ0 */ if (i & 0x01) { diff -uNrp linux-2.6.13/include/asm-i386/mmu_context.h linux-2.6.13-adeos/include/asm-i386/mmu_context.h --- linux-2.6.13/include/asm-i386/mmu_context.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/mmu_context.h 2005-09-04 13:00:26.000000000 +0200 @@ -66,7 +66,17 @@ static inline void switch_mm(struct mm_s #define deactivate_mm(tsk, mm) \ asm("movl %0,%%fs ; movl %0,%%gs": :"r" (0)) +#ifdef CONFIG_ADEOS_CORE +#define activate_mm(prev, next) \ + do { \ + unsigned long flags; \ + adeos_hw_local_irq_save(flags); \ + switch_mm((prev),(next),NULL); \ + adeos_hw_local_irq_restore(flags); \ +} while(0) +#else /* !CONFIG_ADEOS_CORE */ #define activate_mm(prev, next) \ switch_mm((prev),(next),NULL) +#endif /* CONFIG_ADEOS_CORE */ #endif diff -uNrp linux-2.6.13/include/asm-i386/nmi.h linux-2.6.13-adeos/include/asm-i386/nmi.h --- linux-2.6.13/include/asm-i386/nmi.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/nmi.h 2005-10-02 22:21:55.000000000 +0200 @@ -25,4 +25,6 @@ void set_nmi_callback(nmi_callback_t cal */ void unset_nmi_callback(void); +void die_nmi(struct pt_regs *, const char *msg); + #endif /* ASM_NMI_H */ diff -uNrp linux-2.6.13/include/asm-i386/pgalloc.h linux-2.6.13-adeos/include/asm-i386/pgalloc.h --- linux-2.6.13/include/asm-i386/pgalloc.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/pgalloc.h 2005-09-04 12:38:02.000000000 +0200 @@ -47,4 +47,28 @@ static inline void pte_free(struct page #define check_pgt_cache() do { } while (0) +#ifdef CONFIG_ADEOS_CORE +static inline void set_pgdir(unsigned long address, pgd_t entry) +{ + + struct task_struct * p; + pgd_t *pgd; + struct page *page; + + read_lock(&tasklist_lock); + + for_each_process(p) { + if(p->mm) + *pgd_offset(p->mm,address) = entry; + } + + read_unlock(&tasklist_lock); + + for (page = pgd_list; page; page = (struct page *)page->index) { + pgd = (pgd_t *)page_address(page); + pgd[address >> PGDIR_SHIFT] = entry; + } +} +#endif /* CONFIG_ADEOS_CORE */ + #endif /* _I386_PGALLOC_H */ diff -uNrp linux-2.6.13/include/asm-i386/smp.h linux-2.6.13-adeos/include/asm-i386/smp.h --- linux-2.6.13/include/asm-i386/smp.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/smp.h 2005-09-04 12:38:02.000000000 +0200 @@ -55,7 +55,12 @@ extern void cpu_uninit(void); * from the initial startup. We map APIC_BASE very early in page_setup(), * so this is correct in the x86 case. */ +#ifdef CONFIG_ADEOS_CORE +#include +#define raw_smp_processor_id() adeos_processor_id() +#else /* !CONFIG_ADEOS_CORE */ #define raw_smp_processor_id() (current_thread_info()->cpu) +#endif /* CONFIG_ADEOS_CORE */ extern cpumask_t cpu_callout_map; extern cpumask_t cpu_callin_map; diff -uNrp linux-2.6.13/include/asm-i386/spinlock.h linux-2.6.13-adeos/include/asm-i386/spinlock.h --- linux-2.6.13/include/asm-i386/spinlock.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/spinlock.h 2005-09-04 12:38:02.000000000 +0200 @@ -57,6 +57,9 @@ typedef struct { "jmp 1b\n" \ "3:\n\t" +#ifdef CONFIG_ADEOS_CORE +#define spin_lock_string_flags spin_lock_string +#else /* !CONFIG_ADEOS_CORE */ #define spin_lock_string_flags \ "\n1:\t" \ "lock ; decb %0\n\t" \ @@ -72,6 +75,7 @@ typedef struct { "cli\n\t" \ "jmp 1b\n" \ "4:\n\t" +#endif /* CONFIG_ADEOS_CORE */ /* * This works. Despite all the confusion. diff -uNrp linux-2.6.13/include/asm-i386/system.h linux-2.6.13-adeos/include/asm-i386/system.h --- linux-2.6.13/include/asm-i386/system.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/asm-i386/system.h 2005-09-10 12:26:30.000000000 +0200 @@ -441,6 +441,33 @@ struct alt_instr { #define set_wmb(var, value) do { var = value; wmb(); } while (0) /* interrupt control.. */ +#ifdef CONFIG_ADEOS_CORE + +#include + +void __adeos_stall_root(void); + +void __adeos_unstall_root(void); + +unsigned long __adeos_test_root(void); + +unsigned long __adeos_test_and_stall_root(void); + +void fastcall __adeos_restore_root(unsigned long flags); + +#define local_save_flags(x) ((x) = (!__adeos_test_root()) << 9) +#define local_irq_save(x) ((x) = (!__adeos_test_and_stall_root()) << 9) +#define local_irq_restore(x) __adeos_restore_root(!(x & 0x200)) +#define local_irq_disable() __adeos_stall_root() +#define local_irq_enable() __adeos_unstall_root() + +#define irqs_disabled() __adeos_test_root() + +#define safe_halt() \ +__asm__ __volatile__("call __adeos_unstall_root; hlt": : :"memory") + +#else /* !CONFIG_ADEOS_CORE */ + #define local_save_flags(x) do { typecheck(unsigned long,x); __asm__ __volatile__("pushfl ; popl %0":"=g" (x): /* no input */); } while (0) #define local_irq_restore(x) do { typecheck(unsigned long,x); __asm__ __volatile__("pushl %0 ; popfl": /* no output */ :"g" (x):"memory", "cc"); } while (0) #define local_irq_disable() __asm__ __volatile__("cli": : :"memory") @@ -458,6 +485,8 @@ struct alt_instr { /* For spinlocks etc */ #define local_irq_save(x) __asm__ __volatile__("pushfl ; popl %0 ; cli":"=g" (x): /* no input */ :"memory") +#endif /* CONFIG_ADEOS_CORE */ + /* * disable hlt during certain critical i/o operations */ diff -uNrp linux-2.6.13/include/linux/adeos.h linux-2.6.13-adeos/include/linux/adeos.h --- linux-2.6.13/include/linux/adeos.h 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/include/linux/adeos.h 2005-07-24 19:33:25.000000000 +0200 @@ -0,0 +1,553 @@ +/* + * include/linux/adeos.h + * + * Copyright (C) 2002,2003,2004 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#ifndef __LINUX_ADEOS_H +#define __LINUX_ADEOS_H + +#include + +#ifdef CONFIG_ADEOS_CORE + +#include +#include + +#define ADEOS_VERSION_PREFIX "2.6" +#define ADEOS_VERSION_STRING (ADEOS_VERSION_PREFIX ADEOS_ARCH_STRING) +#define ADEOS_RELEASE_NUMBER (0x02060000|((ADEOS_MAJOR_NUMBER&0xff)<<8)|(ADEOS_MINOR_NUMBER&0xff)) + +#define ADEOS_ROOT_PRI 100 +#define ADEOS_ROOT_ID 0 +#define ADEOS_ROOT_NPTDKEYS 4 /* Must be <= 32 */ + +#define ADEOS_RESET_TIMER 0x1 +#define ADEOS_SAME_HANDLER ((void (*)(unsigned))(-1)) + +/* Global domain flags */ +#define ADEOS_SPRINTK_FLAG 0 /* Synchronous printk() allowed */ +#define ADEOS_PPRINTK_FLAG 1 /* Asynchronous printk() request pending */ + +/* Per-cpu pipeline flags. + WARNING: some implementation might refer to those flags + non-symbolically in assembly portions (e.g. x86). */ +#define IPIPE_STALL_FLAG 0 /* Stalls a pipeline stage */ +#define IPIPE_XPEND_FLAG 1 /* Exception notification is pending */ +#define IPIPE_SLEEP_FLAG 2 /* Domain has self-suspended */ +#define IPIPE_SYNC_FLAG 3 /* The interrupt syncer is running for the domain */ + +#define IPIPE_HANDLE_FLAG 0 +#define IPIPE_PASS_FLAG 1 +#define IPIPE_ENABLE_FLAG 2 +#define IPIPE_DYNAMIC_FLAG IPIPE_HANDLE_FLAG +#define IPIPE_EXCLUSIVE_FLAG 3 +#define IPIPE_STICKY_FLAG 4 +#define IPIPE_SYSTEM_FLAG 5 +#define IPIPE_LOCK_FLAG 6 +#define IPIPE_SHARED_FLAG 7 +#define IPIPE_CALLASM_FLAG 8 /* Arch-dependent -- might be unused. */ + +#define IPIPE_HANDLE_MASK (1 << IPIPE_HANDLE_FLAG) +#define IPIPE_PASS_MASK (1 << IPIPE_PASS_FLAG) +#define IPIPE_ENABLE_MASK (1 << IPIPE_ENABLE_FLAG) +#define IPIPE_DYNAMIC_MASK IPIPE_HANDLE_MASK +#define IPIPE_EXCLUSIVE_MASK (1 << IPIPE_EXCLUSIVE_FLAG) +#define IPIPE_STICKY_MASK (1 << IPIPE_STICKY_FLAG) +#define IPIPE_SYSTEM_MASK (1 << IPIPE_SYSTEM_FLAG) +#define IPIPE_LOCK_MASK (1 << IPIPE_LOCK_FLAG) +#define IPIPE_SHARED_MASK (1 << IPIPE_SHARED_FLAG) +#define IPIPE_SYNC_MASK (1 << IPIPE_SYNC_FLAG) +#define IPIPE_CALLASM_MASK (1 << IPIPE_CALLASM_FLAG) + +#define IPIPE_DEFAULT_MASK (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK) + +typedef struct adattr { + + unsigned domid; /* Domain identifier -- Magic value set by caller */ + const char *name; /* Domain name -- Warning: won't be dup'ed! */ + int priority; /* Priority in interrupt pipeline */ + void (*entry)(int); /* Domain entry point */ + int estacksz; /* Stack size for entry context -- 0 means unspec */ + void (*dswitch)(void); /* Handler called each time the domain is switched in */ + int nptdkeys; /* Max. number of per-thread data keys */ + void (*ptdset)(int,void *); /* Routine to set pt values */ + void *(*ptdget)(int); /* Routine to get pt values */ + +} adattr_t; + +typedef struct admutex { + + raw_spinlock_t lock; + +#ifdef CONFIG_ADEOS_THREADS + adomain_t *sleepq, /* Pending domain queue */ + *owner; /* Domain owning the mutex */ +#ifdef CONFIG_SMP + volatile int owncpu; +#define ADEOS_MUTEX_UNLOCKED { RAW_SPIN_LOCK_UNLOCKED, NULL, NULL, -1 } +#else /* !CONFIG_SMP */ +#define ADEOS_MUTEX_UNLOCKED { RAW_SPIN_LOCK_UNLOCKED, NULL, NULL } +#endif /* CONFIG_SMP */ +#else /* !CONFIG_ADEOS_THREADS */ +#define ADEOS_MUTEX_UNLOCKED { RAW_SPIN_LOCK_UNLOCKED } +#endif /* CONFIG_ADEOS_THREADS */ + +} admutex_t; + +typedef void (*adevhand_t)(adevinfo_t *); + +extern int adp_pipelined; + +extern adomain_t *adp_cpu_current[], + *adp_root; + +extern int __adeos_event_monitors[]; + +extern unsigned __adeos_printk_virq; + +extern unsigned long __adeos_virtual_irq_map; + +extern struct list_head __adeos_pipeline; + +extern raw_spinlock_t __adeos_pipelock; + +#ifdef CONFIG_ADEOS_PROFILING + +typedef struct adprofdata { + + struct { + unsigned long long t_handled; + unsigned long long t_synced; + unsigned long n_handled; + unsigned long n_synced; + } irqs[IPIPE_NR_IRQS]; + +} adprofdata_t; + +extern adprofdata_t __adeos_profile_data[ADEOS_NR_CPUS]; + +#endif /* CONFIG_ADEOS_PROFILING */ + +/* Private interface */ + +#ifdef CONFIG_PROC_FS +void __adeos_init_proc(void); +#endif /* CONFIG_PROC_FS */ + +void __adeos_takeover(void); + +asmlinkage int __adeos_handle_event(unsigned event, + void *evdata); + +void __adeos_flush_printk(unsigned irq); + +void __adeos_dump_state(void); + +static inline void __adeos_schedule_head(void *evdata) { + + if (__adeos_event_monitors[ADEOS_SCHEDULE_HEAD] > 0) + __adeos_handle_event(ADEOS_SCHEDULE_HEAD,evdata); +} + +static inline int __adeos_schedule_tail(void *evdata) { + + if (__adeos_event_monitors[ADEOS_SCHEDULE_TAIL] > 0) + return __adeos_handle_event(ADEOS_SCHEDULE_TAIL,evdata); + + return 0; +} + +static inline void __adeos_enter_process(void) { + + if (__adeos_event_monitors[ADEOS_ENTER_PROCESS] > 0) + __adeos_handle_event(ADEOS_ENTER_PROCESS,NULL); +} + +static inline void __adeos_exit_process(void *evdata) { + + if (__adeos_event_monitors[ADEOS_EXIT_PROCESS] > 0) + __adeos_handle_event(ADEOS_EXIT_PROCESS,evdata); +} + +static inline int __adeos_signal_process(void *evdata) { + + if (__adeos_event_monitors[ADEOS_SIGNAL_PROCESS] > 0) + return __adeos_handle_event(ADEOS_SIGNAL_PROCESS,evdata); + + return 0; +} + +static inline void __adeos_kick_process(void *evdata) { + + if (__adeos_event_monitors[ADEOS_KICK_PROCESS] > 0) + __adeos_handle_event(ADEOS_KICK_PROCESS,evdata); +} + +static inline int __adeos_renice_process(void *evdata) { + + if (__adeos_event_monitors[ADEOS_RENICE_PROCESS] > 0) + return __adeos_handle_event(ADEOS_RENICE_PROCESS,evdata); + + return 0; +} + +void __adeos_stall_root(void); + +void __adeos_unstall_root(void); + +unsigned long __adeos_test_root(void); + +unsigned long __adeos_test_and_stall_root(void); + +void fastcall __adeos_restore_root(unsigned long flags); + +void __adeos_schedule_back_root(struct task_struct *prev); + +int __adeos_setscheduler_root(struct task_struct *p, + int policy, + int prio); + +void __adeos_reenter_root(struct task_struct *prev, + int policy, + int prio); + +int fastcall __adeos_schedule_irq(unsigned irq, + struct list_head *head); + +#define __adeos_pipeline_head_p(adp) (&(adp)->p_link == __adeos_pipeline.next) + +#ifdef CONFIG_ADEOS_THREADS + +static inline int __adeos_domain_work_p (adomain_t *adp, int cpuid) + +{ + return (!test_bit(IPIPE_SLEEP_FLAG,&adp->cpudata[cpuid].status) || + (!test_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status) && + adp->cpudata[cpuid].irq_pending_hi != 0) || + test_bit(IPIPE_XPEND_FLAG,&adp->cpudata[cpuid].status)); +} + +#else /* !CONFIG_ADEOS_THREADS */ + +static inline int __adeos_domain_work_p (adomain_t *adp, int cpuid) + +{ + return (!test_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status) && + adp->cpudata[cpuid].irq_pending_hi != 0); +} + +static inline void __adeos_switch_to (adomain_t *out, adomain_t *in, int cpuid) + +{ + void adeos_suspend_domain(void); + + /* "in" is guaranteed to be closer than "out" from the head of the + pipeline (and obviously different). */ + + adp_cpu_current[cpuid] = in; + + if (in->dswitch) + in->dswitch(); + + adeos_suspend_domain(); /* Sync stage and propagate interrupts. */ + adeos_load_cpuid(); /* Processor might have changed. */ + + if (adp_cpu_current[cpuid] == in) + /* Otherwise, something has changed the current domain under + our feet recycling the register set; do not override. */ + adp_cpu_current[cpuid] = out; +} + +#endif /* CONFIG_ADEOS_THREADS */ + +/* Public interface */ + +int adeos_register_domain(adomain_t *adp, + adattr_t *attr); + +int adeos_unregister_domain(adomain_t *adp); + +void adeos_suspend_domain(void); + +int adeos_virtualize_irq_from(adomain_t *adp, + unsigned irq, + void (*handler)(unsigned irq), + int (*acknowledge)(unsigned irq), + unsigned modemask); + +static inline int adeos_virtualize_irq(unsigned irq, + void (*handler)(unsigned irq), + int (*acknowledge)(unsigned irq), + unsigned modemask) { + + return adeos_virtualize_irq_from(adp_current, + irq, + handler, + acknowledge, + modemask); +} + +int adeos_control_irq(unsigned irq, + unsigned clrmask, + unsigned setmask); + +cpumask_t adeos_set_irq_affinity(unsigned irq, + cpumask_t cpumask); + +static inline int adeos_share_irq (unsigned irq, int (*acknowledge)(unsigned irq)) { + + return adeos_virtualize_irq(irq, + ADEOS_SAME_HANDLER, + acknowledge, + IPIPE_SHARED_MASK|IPIPE_HANDLE_MASK|IPIPE_PASS_MASK); +} + +unsigned adeos_alloc_irq(void); + +int adeos_free_irq(unsigned irq); + +int fastcall adeos_trigger_irq(unsigned irq); + +static inline int adeos_propagate_irq(unsigned irq) { + + return __adeos_schedule_irq(irq,adp_current->p_link.next); +} + +static inline int adeos_schedule_irq(unsigned irq) { + + return __adeos_schedule_irq(irq,&adp_current->p_link); +} + +int fastcall adeos_send_ipi(unsigned ipi, + cpumask_t cpumask); + +static inline void adeos_stall_pipeline_from (adomain_t *adp) + +{ + adeos_declare_cpuid; +#ifdef CONFIG_SMP + unsigned long flags; + + adeos_lock_cpu(flags); + + __set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + if (!__adeos_pipeline_head_p(adp)) + adeos_unlock_cpu(flags); +#else /* CONFIG_SMP */ + set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + if (__adeos_pipeline_head_p(adp)) + adeos_hw_cli(); +#endif /* CONFIG_SMP */ +} + +static inline unsigned long adeos_test_pipeline_from (adomain_t *adp) + +{ + unsigned long flags, s; + adeos_declare_cpuid; + + adeos_get_cpu(flags); + s = test_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + adeos_put_cpu(flags); + + return s; +} + +static inline unsigned long adeos_test_and_stall_pipeline_from (adomain_t *adp) + +{ + adeos_declare_cpuid; + unsigned long s; +#ifdef CONFIG_SMP + unsigned long flags; + + adeos_lock_cpu(flags); + + s = __test_and_set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + if (!__adeos_pipeline_head_p(adp)) + adeos_unlock_cpu(flags); +#else /* CONFIG_SMP */ + s = test_and_set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + if (__adeos_pipeline_head_p(adp)) + adeos_hw_cli(); +#endif /* CONFIG_SMP */ + + return s; +} + +void fastcall adeos_unstall_pipeline_from(adomain_t *adp); + +static inline unsigned long adeos_test_and_unstall_pipeline_from(adomain_t *adp) + +{ + unsigned long flags, s; + adeos_declare_cpuid; + + adeos_get_cpu(flags); + s = test_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + adeos_unstall_pipeline_from(adp); + adeos_put_cpu(flags); + + return s; +} + +static inline void adeos_unstall_pipeline(void) + +{ + adeos_unstall_pipeline_from(adp_current); +} + +static inline unsigned long adeos_test_and_unstall_pipeline(void) + +{ + return adeos_test_and_unstall_pipeline_from(adp_current); +} + +static inline unsigned long adeos_test_pipeline (void) + +{ + return adeos_test_pipeline_from(adp_current); +} + +static inline unsigned long adeos_test_and_stall_pipeline (void) + +{ + return adeos_test_and_stall_pipeline_from(adp_current); +} + +static inline void adeos_restore_pipeline_from (adomain_t *adp, unsigned long flags) + +{ + if (flags) + adeos_stall_pipeline_from(adp); + else + adeos_unstall_pipeline_from(adp); +} + +static inline void adeos_stall_pipeline (void) + +{ + adeos_stall_pipeline_from(adp_current); +} + +static inline void adeos_restore_pipeline (unsigned long flags) + +{ + adeos_restore_pipeline_from(adp_current,flags); +} + +static inline void adeos_restore_pipeline_nosync (adomain_t *adp, unsigned long flags, int cpuid) + +{ + /* If cpuid is current, then it must be held on entry + (adeos_get_cpu/adeos_hw_local_irq_save/adeos_hw_cli). */ + + if (flags) + __set_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + else + __clear_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); +} + +adevhand_t adeos_catch_event_from(adomain_t *adp, + unsigned event, + adevhand_t handler); + +static inline adevhand_t adeos_catch_event (unsigned event, adevhand_t handler) + +{ + return adeos_catch_event_from(adp_current,event,handler); +} + +static inline void adeos_propagate_event(adevinfo_t *evinfo) + +{ + evinfo->propagate = 1; +} + +void adeos_init_attr(adattr_t *attr); + +int adeos_get_sysinfo(adsysinfo_t *sysinfo); + +int adeos_tune_timer(unsigned long ns, + int flags); + +int adeos_alloc_ptdkey(void); + +int adeos_free_ptdkey(int key); + +int adeos_set_ptd(int key, + void *value); + +void *adeos_get_ptd(int key); + +unsigned long adeos_critical_enter(void (*syncfn)(void)); + +void adeos_critical_exit(unsigned long flags); + +int adeos_init_mutex(admutex_t *mutex); + +int adeos_destroy_mutex(admutex_t *mutex); + +unsigned long fastcall adeos_lock_mutex(admutex_t *mutex); + +void fastcall adeos_unlock_mutex(admutex_t *mutex, + unsigned long flags); + +static inline void adeos_set_printk_sync (adomain_t *adp) { + set_bit(ADEOS_SPRINTK_FLAG,&adp->flags); +} + +static inline void adeos_set_printk_async (adomain_t *adp) { + clear_bit(ADEOS_SPRINTK_FLAG,&adp->flags); +} + +#define spin_lock_irqsave_hw_cond(lock,flags) spin_lock_irqsave_hw(lock,flags) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock_irqrestore_hw(lock,flags) + +#define pic_irq_lock(irq) \ + do { \ + adeos_declare_cpuid; \ + adeos_load_cpuid(); \ + __adeos_lock_irq(adp_cpu_current[cpuid], cpuid, irq); \ + } while(0) + +#define pic_irq_unlock(irq) \ + do { \ + adeos_declare_cpuid; \ + adeos_load_cpuid(); \ + __adeos_unlock_irq(adp_cpu_current[cpuid], irq); \ + } while(0) + +#else /* !CONFIG_ADEOS_CORE */ + +#define spin_lock_irqsave_hw(lock,flags) spin_lock_irqsave(lock, flags) +#define spin_unlock_irqrestore_hw(lock,flags) spin_unlock_irqrestore(lock, flags) +#define spin_lock_irqsave_hw_cond(lock,flags) do { flags = 0; spin_lock(lock); } while(0) +#define spin_unlock_irqrestore_hw_cond(lock,flags) spin_unlock(lock) + +#define pic_irq_lock(irq) do { } while(0) +#define pic_irq_unlock(irq) do { } while(0) + +#endif /* CONFIG_ADEOS_CORE */ + +#endif /* !__LINUX_ADEOS_H */ diff -uNrp linux-2.6.13/include/linux/preempt.h linux-2.6.13-adeos/include/linux/preempt.h --- linux-2.6.13/include/linux/preempt.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/linux/preempt.h 2005-09-04 12:38:02.000000000 +0200 @@ -26,6 +26,47 @@ asmlinkage void preempt_schedule(void); +#ifdef CONFIG_ADEOS_CORE + +#include + +extern adomain_t *adp_cpu_current[], + *adp_root; + +#define preempt_disable() \ +do { \ + if (adp_current == adp_root) { \ + inc_preempt_count(); \ + barrier(); \ + } \ +} while (0) + +#define preempt_enable_no_resched() \ +do { \ + if (adp_current == adp_root) { \ + barrier(); \ + dec_preempt_count(); \ + } \ +} while (0) + +#define preempt_check_resched() \ +do { \ + if (adp_current == adp_root) { \ + if (unlikely(test_thread_flag(TIF_NEED_RESCHED))) \ + preempt_schedule(); \ + } \ +} while (0) + +#define preempt_enable() \ +do { \ + if (adp_current == adp_root) { \ + preempt_enable_no_resched(); \ + preempt_check_resched(); \ + } \ +} while (0) + +#else /* !CONFIG_ADEOS_CORE */ + #define preempt_disable() \ do { \ inc_preempt_count(); \ @@ -50,6 +91,8 @@ do { \ preempt_check_resched(); \ } while (0) +#endif /* CONFIG_ADEOS_CORE */ + #else #define preempt_disable() do { } while (0) diff -uNrp linux-2.6.13/include/linux/sched.h linux-2.6.13-adeos/include/linux/sched.h --- linux-2.6.13/include/linux/sched.h 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/include/linux/sched.h 2005-09-04 12:38:02.000000000 +0200 @@ -4,6 +4,7 @@ #include /* for HZ */ #include +#include #include #include #include @@ -770,6 +771,9 @@ struct task_struct { int cpuset_mems_generation; #endif atomic_t fs_excl; /* holding fs exclusive resources */ +#ifdef CONFIG_ADEOS_CORE + void *ptd[ADEOS_ROOT_NPTDKEYS]; +#endif /* CONFIG_ADEOS_CORE */ }; static inline pid_t process_group(struct task_struct *tsk) diff -uNrp linux-2.6.13/init/Kconfig linux-2.6.13-adeos/init/Kconfig --- linux-2.6.13/init/Kconfig 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/init/Kconfig 2005-09-04 12:38:02.000000000 +0200 @@ -69,6 +69,7 @@ menu "General setup" config LOCALVERSION string "Local version - append to kernel release" + default "-adeos" help Append an extra string to the end of your kernel version. This will show up when you type uname, for example. diff -uNrp linux-2.6.13/init/main.c linux-2.6.13-adeos/init/main.c --- linux-2.6.13/init/main.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/init/main.c 2005-09-04 12:38:02.000000000 +0200 @@ -470,6 +470,9 @@ asmlinkage void __init start_kernel(void trap_init(); rcu_init(); init_IRQ(); +#ifdef CONFIG_ADEOS_CORE + __adeos_init(); +#endif /* CONFIG_ADEOS_CORE */ pidhash_init(); init_timers(); softirq_init(); @@ -603,6 +606,10 @@ static void __init do_basic_setup(void) sock_init(); do_initcalls(); + +#ifdef CONFIG_ADEOS + __adeos_takeover(); +#endif /* CONFIG_ADEOS */ } static void do_pre_smp_initcalls(void) diff -uNrp linux-2.6.13/kernel/Makefile linux-2.6.13-adeos/kernel/Makefile --- linux-2.6.13/kernel/Makefile 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/Makefile 2005-09-04 12:38:02.000000000 +0200 @@ -9,6 +9,7 @@ obj-y = sched.o fork.o exec_domain.o rcupdate.o intermodule.o extable.o params.o posix-timers.o \ kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o +obj-$(CONFIG_ADEOS_CORE) += adeos.o obj-$(CONFIG_FUTEX) += futex.o obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o obj-$(CONFIG_SMP) += cpu.o spinlock.o diff -uNrp linux-2.6.13/kernel/adeos.c linux-2.6.13-adeos/kernel/adeos.c --- linux-2.6.13/kernel/adeos.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.6.13-adeos/kernel/adeos.c 2005-09-10 19:20:07.000000000 +0200 @@ -0,0 +1,828 @@ +/* + * linux/kernel/adeos.c + * + * Copyright (C) 2002,2003,2004 Philippe Gerum. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA 02139, + * USA; either version 2 of the License, or (at your option) any later + * version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Architecture-independent ADEOS core support. + */ + +#include +#include +#ifdef CONFIG_PROC_FS +#include +#endif /* CONFIG_PROC_FS */ + +/* The pre-defined domain slot for the root domain. */ +static adomain_t adeos_root_domain; + +/* A constant pointer to the root domain. */ +adomain_t *adp_root = &adeos_root_domain; + +/* A pointer to the current domain. */ +adomain_t *adp_cpu_current[ADEOS_NR_CPUS] = { [ 0 ... ADEOS_NR_CPUS - 1] = &adeos_root_domain }; + +/* The spinlock protecting from races while modifying the pipeline. */ +raw_spinlock_t __adeos_pipelock = RAW_SPIN_LOCK_UNLOCKED; + +/* The pipeline data structure. Enqueues adomain_t objects by priority. */ +struct list_head __adeos_pipeline; + +/* A global flag telling whether Adeos pipelining is engaged. */ +int adp_pipelined; + +/* An array of global counters tracking domains monitoring events. */ +int __adeos_event_monitors[ADEOS_NR_EVENTS] = { [ 0 ... ADEOS_NR_EVENTS - 1] = 0 }; + +/* The allocated VIRQ map. */ +unsigned long __adeos_virtual_irq_map = 0; + +/* A VIRQ to kick printk() output out when the root domain is in control. */ +unsigned __adeos_printk_virq; + +#ifdef CONFIG_ADEOS_PROFILING +adprofdata_t __adeos_profile_data[ADEOS_NR_CPUS]; +#endif /* CONFIG_ADEOS_PROFILING */ + +static void __adeos_set_root_ptd (int key, void *value) { + + current->ptd[key] = value; +} + +static void *__adeos_get_root_ptd (int key) { + + return current->ptd[key]; +} + +/* adeos_init() -- Initialization routine of the ADEOS layer. Called + by the host kernel early during the boot procedure. */ + +void __adeos_init (void) + +{ + adomain_t *adp = &adeos_root_domain; + + __adeos_check_platform(); /* Do platform dependent checks first. */ + + /* + A lightweight registration code for the root domain. Current + assumptions are: + - We are running on the boot CPU, and secondary CPUs are still + lost in space. + - adeos_root_domain has been zero'ed. + */ + + INIT_LIST_HEAD(&__adeos_pipeline); + + adp->name = "Linux"; + adp->domid = ADEOS_ROOT_ID; + adp->priority = ADEOS_ROOT_PRI; + adp->ptd_setfun = &__adeos_set_root_ptd; + adp->ptd_getfun = &__adeos_get_root_ptd; + adp->ptd_keymax = ADEOS_ROOT_NPTDKEYS; + + __adeos_init_stage(adp); + + INIT_LIST_HEAD(&adp->p_link); + list_add_tail(&adp->p_link,&__adeos_pipeline); + + __adeos_init_platform(); + + __adeos_printk_virq = adeos_alloc_irq(); /* Cannot fail here. */ + adp->irqs[__adeos_printk_virq].handler = &__adeos_flush_printk; + adp->irqs[__adeos_printk_virq].acknowledge = NULL; + adp->irqs[__adeos_printk_virq].control = IPIPE_HANDLE_MASK; + + printk(KERN_INFO "Adeos %s: Root domain %s registered.\n", + ADEOS_VERSION_STRING, + adp->name); +} + +/* adeos_handle_event() -- Adeos' generic event handler. This routine + calls the per-domain handlers registered for a given + exception/event. Each domain before the one which raised the event + in the pipeline will get a chance to process the event. The latter + will eventually be allowed to process its own event too if a valid + handler exists for it. Handler executions are always scheduled by + the domain which raised the event for the higher priority domains + wanting to be notified of such event. Note: evdata might be + NULL. */ + +#ifdef CONFIG_ADEOS_THREADS + +asmlinkage int __adeos_handle_event (unsigned event, void *evdata) +/* asmlinkage is there just in case CONFIG_REGPARM is enabled... */ +{ + struct list_head *pos, *npos; + adomain_t *this_domain; + unsigned long flags; + adeos_declare_cpuid; + adevinfo_t evinfo; + int propagate = 1; + + adeos_lock_cpu(flags); + + this_domain = adp_cpu_current[cpuid]; + + list_for_each_safe(pos,npos,&__adeos_pipeline) { + + adomain_t *next_domain = list_entry(pos,adomain_t,p_link); + + if (next_domain->events[event].handler != NULL) + { + if (next_domain == this_domain) + { + adeos_unlock_cpu(flags); + evinfo.domid = this_domain->domid; + evinfo.event = event; + evinfo.evdata = evdata; + evinfo.propagate = 0; + this_domain->events[event].handler(&evinfo); + propagate = evinfo.propagate; + goto done; + } + + next_domain->cpudata[cpuid].event_info.domid = this_domain->domid; + next_domain->cpudata[cpuid].event_info.event = event; + next_domain->cpudata[cpuid].event_info.evdata = evdata; + next_domain->cpudata[cpuid].event_info.propagate = 0; + __set_bit(IPIPE_XPEND_FLAG,&next_domain->cpudata[cpuid].status); + + /* Let the higher priority domain process the event. */ + __adeos_switch_to(this_domain,next_domain,cpuid); + + adeos_load_cpuid(); /* Processor might have changed. */ + + if (!next_domain->cpudata[cpuid].event_info.propagate) + { + propagate = 0; + break; + } + } + + if (next_domain != adp_root && /* NEVER sync the root stage here. */ + next_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) + { + if (next_domain != this_domain) + __adeos_switch_to(this_domain,next_domain,cpuid); + else + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + + adeos_load_cpuid(); /* Processor might have changed. */ + } + + if (next_domain == this_domain) + break; + } + + adeos_unlock_cpu(flags); + + done: + + return !propagate; +} + +#else /* !CONFIG_ADEOS_THREADS */ + +asmlinkage int __adeos_handle_event (unsigned event, void *evdata) +/* asmlinkage is there just in case CONFIG_REGPARM is enabled... */ +{ + adomain_t *start_domain, *this_domain, *next_domain; + struct list_head *pos, *npos; + unsigned long flags; + adeos_declare_cpuid; + adevinfo_t evinfo; + int propagate = 1; + + adeos_lock_cpu(flags); + + start_domain = this_domain = adp_cpu_current[cpuid]; + + list_for_each_safe(pos,npos,&__adeos_pipeline) { + + next_domain = list_entry(pos,adomain_t,p_link); + + /* Note: Domain migration may occur while running event or + interrupt handlers, in which case the current register set + is going to be recycled for a different domain than the + initiating one. We do care for that, always tracking the + current domain descriptor upon return from those + handlers. */ + + if (next_domain->events[event].handler != NULL) + { + adp_cpu_current[cpuid] = next_domain; + evinfo.domid = start_domain->domid; + adeos_unlock_cpu(flags); + evinfo.event = event; + evinfo.evdata = evdata; + evinfo.propagate = 0; + next_domain->events[event].handler(&evinfo); + adeos_lock_cpu(flags); + + if (adp_cpu_current[cpuid] != next_domain) + this_domain = adp_cpu_current[cpuid]; + + propagate = evinfo.propagate; + } + + if (next_domain != adp_root && /* NEVER sync the root stage here. */ + next_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) + { + adp_cpu_current[cpuid] = next_domain; + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + adeos_load_cpuid(); + + if (adp_cpu_current[cpuid] != next_domain) + this_domain = adp_cpu_current[cpuid]; + } + + adp_cpu_current[cpuid] = this_domain; + + if (next_domain == this_domain || !propagate) + break; + } + + adeos_unlock_cpu(flags); + + return !propagate; +} + +#endif /* CONFIG_ADEOS_THREADS */ + +void __adeos_stall_root (void) + +{ + if (adp_pipelined) + { + adeos_declare_cpuid; + +#ifdef CONFIG_SMP + unsigned long flags; + adeos_lock_cpu(flags); + __set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + adeos_unlock_cpu(flags); +#else /* !CONFIG_SMP */ + set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); +#endif /* CONFIG_SMP */ + } + else + adeos_hw_cli(); +} + +void __adeos_unstall_root (void) + +{ + if (adp_pipelined) + { + adeos_declare_cpuid; + + adeos_hw_cli(); + + adeos_load_cpuid(); + + __clear_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + + if (adp_root->cpudata[cpuid].irq_pending_hi != 0) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + } + + adeos_hw_sti(); /* Needed in both cases. */ +} + +unsigned long __adeos_test_root (void) + +{ + if (adp_pipelined) + { + adeos_declare_cpuid; + unsigned long s; + +#ifdef CONFIG_SMP + unsigned long flags; + adeos_lock_cpu(flags); + s = test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + adeos_unlock_cpu(flags); +#else /* !CONFIG_SMP */ + s = test_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); +#endif /* CONFIG_SMP */ + + return s; + } + + return adeos_hw_irqs_disabled(); +} + +unsigned long __adeos_test_and_stall_root (void) + +{ + unsigned long flags; + + if (adp_pipelined) + { + adeos_declare_cpuid; + unsigned long s; + +#ifdef CONFIG_SMP + adeos_lock_cpu(flags); + s = __test_and_set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); + adeos_unlock_cpu(flags); +#else /* !CONFIG_SMP */ + s = test_and_set_bit(IPIPE_STALL_FLAG,&adp_root->cpudata[cpuid].status); +#endif /* CONFIG_SMP */ + + return s; + } + + adeos_hw_local_irq_save(flags); + + return !adeos_hw_test_iflag(flags); +} + +void fastcall __adeos_restore_root (unsigned long flags) + +{ + if (flags) + __adeos_stall_root(); + else + __adeos_unstall_root(); +} + +/* adeos_unstall_pipeline_from() -- Unstall the interrupt pipeline and + synchronize pending events from a given domain. */ + +void fastcall adeos_unstall_pipeline_from (adomain_t *adp) + +{ + adomain_t *this_domain; + struct list_head *pos; + unsigned long flags; + adeos_declare_cpuid; + + adeos_lock_cpu(flags); + + __clear_bit(IPIPE_STALL_FLAG,&adp->cpudata[cpuid].status); + + this_domain = adp_cpu_current[cpuid]; + + if (adp == this_domain) + { + if (adp->cpudata[cpuid].irq_pending_hi != 0) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + + goto release_cpu_and_exit; + } + + /* Attempt to flush all events that might be pending at the + unstalled domain level. This code is roughly lifted from + __adeos_walk_pipeline(). */ + + list_for_each(pos,&__adeos_pipeline) { + + adomain_t *next_domain = list_entry(pos,adomain_t,p_link); + + if (test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) + break; /* Stalled stage -- do not go further. */ + + if (next_domain->cpudata[cpuid].irq_pending_hi != 0) + { + /* Since the critical IPI might be triggered by the + following actions, the current domain might not be + linked to the pipeline anymore after its handler + returns on SMP boxen, even if the domain remains valid + (see adeos_unregister_domain()), so don't make any + hazardous assumptions here. */ + + if (next_domain == this_domain) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + else + { + __adeos_switch_to(this_domain,next_domain,cpuid); + + adeos_load_cpuid(); /* Processor might have changed. */ + + if (this_domain->cpudata[cpuid].irq_pending_hi != 0 && + !test_bit(IPIPE_STALL_FLAG,&this_domain->cpudata[cpuid].status)) + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + } + + break; + } + else if (next_domain == this_domain) + break; + } + +release_cpu_and_exit: + + if (__adeos_pipeline_head_p(adp)) + adeos_hw_sti(); + else + adeos_unlock_cpu(flags); +} + +/* adeos_suspend_domain() -- tell the ADEOS layer that the current + domain is now dormant. The calling domain is switched out, while + the next domain with work in progress or pending in the pipeline is + switched in. */ + +#ifdef CONFIG_ADEOS_THREADS + +#define __flush_pipeline_stage() \ +do { \ + if (!test_bit(IPIPE_STALL_FLAG,&cpudata->status) && \ + cpudata->irq_pending_hi != 0) \ + { \ + __adeos_sync_stage(IPIPE_IRQMASK_ANY); \ + adeos_load_cpuid(); \ + cpudata = &this_domain->cpudata[cpuid]; \ + } \ +} while(0) + +void adeos_suspend_domain (void) + +{ + adomain_t *this_domain, *next_domain; + struct adcpudata *cpudata; + struct list_head *ln; + unsigned long flags; + adeos_declare_cpuid; + + adeos_lock_cpu(flags); + + this_domain = next_domain = adp_cpu_current[cpuid]; + cpudata = &this_domain->cpudata[cpuid]; + + /* A suspending domain implicitely unstalls the pipeline. */ + __clear_bit(IPIPE_STALL_FLAG,&cpudata->status); + + /* Make sure that no event remains stuck in the pipeline. This + could happen with emerging SMP instances, or domains which + forget to unstall their stage before calling us. */ + __flush_pipeline_stage(); + + for (;;) + { + ln = next_domain->p_link.next; + + if (ln == &__adeos_pipeline) /* End of pipeline reached? */ + /* Caller should loop on its idle task on return. */ + goto release_cpu_and_exit; + + next_domain = list_entry(ln,adomain_t,p_link); + + /* Make sure the domain was preempted (i.e. not sleeping) or + has some event to process before switching to it. */ + + if (__adeos_domain_work_p(next_domain,cpuid)) + break; + } + + /* Mark the outgoing domain as aslept (i.e. not preempted). */ + __set_bit(IPIPE_SLEEP_FLAG,&cpudata->status); + + /* Suspend the calling domain, switching to the next one. */ + __adeos_switch_to(this_domain,next_domain,cpuid); + +#ifdef CONFIG_SMP + adeos_load_cpuid(); /* Processor might have changed. */ + cpudata = &this_domain->cpudata[cpuid]; +#endif /* CONFIG_SMP */ + + /* Clear the sleep bit for the incoming domain. */ + __clear_bit(IPIPE_SLEEP_FLAG,&cpudata->status); + + /* Now, we are back into the calling domain. Flush the interrupt + log and fire the event interposition handler if needed. CPU + migration is allowed in SMP-mode on behalf of an event handler + provided that the current domain raised it. Otherwise, it's + not. */ + + __flush_pipeline_stage(); + + if (__test_and_clear_bit(IPIPE_XPEND_FLAG,&cpudata->status)) + { + adeos_unlock_cpu(flags); + this_domain->events[cpudata->event_info.event].handler(&cpudata->event_info); + return; + } + +release_cpu_and_exit: + + adeos_unlock_cpu(flags); + + /* Return to the point of suspension in the calling domain. */ +} + +#else /* !CONFIG_ADEOS_THREADS */ + +void adeos_suspend_domain (void) + +{ + adomain_t *this_domain, *next_domain; + struct list_head *ln; + unsigned long flags; + adeos_declare_cpuid; + + adeos_lock_cpu(flags); + + this_domain = next_domain = adp_cpu_current[cpuid]; + + __clear_bit(IPIPE_STALL_FLAG,&this_domain->cpudata[cpuid].status); + + if (this_domain->cpudata[cpuid].irq_pending_hi != 0) + goto sync_stage; + + for (;;) + { + ln = next_domain->p_link.next; + + if (ln == &__adeos_pipeline) + break; + + next_domain = list_entry(ln,adomain_t,p_link); + + if (test_bit(IPIPE_STALL_FLAG,&next_domain->cpudata[cpuid].status)) + break; + + if (next_domain->cpudata[cpuid].irq_pending_hi == 0) + continue; + + adp_cpu_current[cpuid] = next_domain; + + if (next_domain->dswitch) + next_domain->dswitch(); + + sync_stage: + + __adeos_sync_stage(IPIPE_IRQMASK_ANY); + + adeos_load_cpuid(); /* Processor might have changed. */ + + if (adp_cpu_current[cpuid] != next_domain) + /* Something has changed the current domain under our feet + recycling the register set; take note. */ + this_domain = adp_cpu_current[cpuid]; + } + + adp_cpu_current[cpuid] = this_domain; + + adeos_unlock_cpu(flags); +} + +#endif /* CONFIG_ADEOS_THREADS */ + +/* adeos_alloc_irq() -- Allocate a virtual/soft pipelined interrupt. + Virtual interrupts are handled in exactly the same way than their + hw-generated counterparts. This is a very basic, one-way only, + inter-domain communication system (see adeos_trigger_irq()). Note: + it is not necessary for a domain to allocate a virtual interrupt to + trap it using adeos_virtualize_irq(). The newly allocated VIRQ + number which can be passed to other IRQ-related services is + returned on success, zero otherwise (i.e. no more virtual interrupt + channel is available). We need this service as part of the Adeos + bootstrap code, hence it must reside in a built-in area. */ + +unsigned adeos_alloc_irq (void) + +{ + unsigned long flags, irq = 0; + int ipos; + + spin_lock_irqsave_hw(&__adeos_pipelock,flags); + + if (__adeos_virtual_irq_map != ~0) + { + ipos = ffz(__adeos_virtual_irq_map); + set_bit(ipos,&__adeos_virtual_irq_map); + irq = ipos + IPIPE_VIRQ_BASE; + } + + spin_unlock_irqrestore_hw(&__adeos_pipelock,flags); + + return irq; +} + +#ifdef CONFIG_PROC_FS + +#include + +static struct proc_dir_entry *adeos_proc_entry; + +static int __adeos_read_proc (char *page, + char **start, + off_t off, + int count, + int *eof, + void *data) +{ + unsigned long ctlbits; + struct list_head *pos; + unsigned irq, _irq; + char *p = page; + int len; + +#ifdef CONFIG_ADEOS_MODULE + p += sprintf(p,"Adeos %s -- Pipelining: %s",ADEOS_VERSION_STRING,adp_pipelined ? "active" : "stopped"); +#else /* !CONFIG_ADEOS_MODULE */ + p += sprintf(p,"Adeos %s -- Pipelining: permanent",ADEOS_VERSION_STRING); +#endif /* CONFIG_ADEOS_MODULE */ +#ifdef CONFIG_ADEOS_THREADS + p += sprintf(p, " (threaded)\n\n"); +#else /* CONFIG_ADEOS_THREADS */ + p += sprintf(p, "\n\n"); +#endif /* CONFIG_ADEOS_THREADS */ + + spin_lock(&__adeos_pipelock); + + list_for_each(pos,&__adeos_pipeline) { + + adomain_t *adp = list_entry(pos,adomain_t,p_link); + + p += sprintf(p,"%8s: priority=%d, id=0x%.8x, ptdkeys=%d/%d\n", + adp->name, + adp->priority, + adp->domid, + adp->ptd_keycount, + adp->ptd_keymax); + irq = 0; + + while (irq < IPIPE_NR_IRQS) + { + ctlbits = (adp->irqs[irq].control & (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK|IPIPE_STICKY_MASK)); + + if (irq >= IPIPE_NR_XIRQS && !adeos_virtual_irq_p(irq)) + { + /* There might be a hole between the last external IRQ + and the first virtual one; skip it. */ + irq++; + continue; + } + + if (adeos_virtual_irq_p(irq) && !test_bit(irq - IPIPE_VIRQ_BASE,&__adeos_virtual_irq_map)) + { + /* Non-allocated virtual IRQ; skip it. */ + irq++; + continue; + } + + /* Attempt to group consecutive IRQ numbers having the + same virtualization settings in a single line. */ + + _irq = irq; + + while (++_irq < IPIPE_NR_IRQS) + { + if (adeos_virtual_irq_p(_irq) != adeos_virtual_irq_p(irq) || + (adeos_virtual_irq_p(_irq) && + !test_bit(_irq - IPIPE_VIRQ_BASE,&__adeos_virtual_irq_map)) || + ctlbits != (adp->irqs[_irq].control & (IPIPE_HANDLE_MASK|IPIPE_PASS_MASK|IPIPE_STICKY_MASK))) + break; + } + + if (_irq == irq + 1) + p += sprintf(p,"\tirq%u: ",irq); + else + p += sprintf(p,"\tirq%u-%u: ",irq,_irq - 1); + + /* Statuses are as follows: + o "accepted" means handled _and_ passed down the + pipeline. + o "grabbed" means handled, but the interrupt might be + terminated _or_ passed down the pipeline depending on + what the domain handler asks for to Adeos. + o "passed" means unhandled by the domain but passed + down the pipeline. + o "discarded" means unhandled and _not_ passed down the + pipeline. The interrupt merely disappears from the + current domain down to the end of the pipeline. */ + + if (ctlbits & IPIPE_HANDLE_MASK) + { + if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p,"accepted"); + else + p += sprintf(p,"grabbed"); + } + else if (ctlbits & IPIPE_PASS_MASK) + p += sprintf(p,"passed"); + else + p += sprintf(p,"discarded"); + + if (ctlbits & IPIPE_STICKY_MASK) + p += sprintf(p,", sticky"); + + if (adeos_virtual_irq_p(irq)) + p += sprintf(p,", virtual"); + + p += sprintf(p,"\n"); + + irq = _irq; + } + } + + spin_unlock(&__adeos_pipelock); + + len = p - page; + + if (len <= off + count) + *eof = 1; + + *start = page + off; + + len -= off; + + if (len > count) + len = count; + + if (len < 0) + len = 0; + + return len; +} + +void __adeos_init_proc (void) { + + adeos_proc_entry = create_proc_read_entry("adeos", + 0444, + NULL, + &__adeos_read_proc, + NULL); +} + +#endif /* CONFIG_PROC_FS */ + +void __adeos_dump_state (void) + +{ + int _cpuid, nr_cpus = num_online_cpus(); + struct list_head *pos; + unsigned long flags; + adeos_declare_cpuid; + + adeos_lock_cpu(flags); + + printk(KERN_WARNING "Adeos: Current domain=%s on CPU #%d [stackbase=%p]\n", + adp_current->name, + cpuid, +#ifdef CONFIG_ADEOS_THREADS + (void *)adp_current->estackbase[cpuid] +#else /* !CONFIG_ADEOS_THREADS */ + current +#endif /* CONFIG_ADEOS_THREADS */ + ); + + list_for_each(pos,&__adeos_pipeline) { + + adomain_t *adp = list_entry(pos,adomain_t,p_link); + + for (_cpuid = 0; _cpuid < nr_cpus; _cpuid++) + printk(KERN_WARNING "%8s[cpuid=%d]: priority=%d, status=0x%lx, pending_hi=0x%lx\n", + adp->name, + _cpuid, + adp->priority, + adp->cpudata[_cpuid].status, + adp->cpudata[_cpuid].irq_pending_hi); + } + + adeos_unlock_cpu(flags); +} + +EXPORT_SYMBOL(adeos_suspend_domain); +EXPORT_SYMBOL(adeos_alloc_irq); +EXPORT_SYMBOL(adp_cpu_current); +EXPORT_SYMBOL(adp_root); +EXPORT_SYMBOL(adp_pipelined); +EXPORT_SYMBOL(__adeos_handle_event); +EXPORT_SYMBOL(__adeos_unstall_root); +EXPORT_SYMBOL(__adeos_stall_root); +EXPORT_SYMBOL(__adeos_restore_root); +EXPORT_SYMBOL(__adeos_test_and_stall_root); +EXPORT_SYMBOL(__adeos_test_root); +EXPORT_SYMBOL(__adeos_dump_state); +EXPORT_SYMBOL(__adeos_pipeline); +EXPORT_SYMBOL(__adeos_pipelock); +EXPORT_SYMBOL(__adeos_virtual_irq_map); +EXPORT_SYMBOL(__adeos_event_monitors); +EXPORT_SYMBOL(adeos_unstall_pipeline_from); +#ifdef CONFIG_ADEOS_PROFILING +EXPORT_SYMBOL(__adeos_profile_data); +#endif /* CONFIG_ADEOS_PROFILING */ +/* The following are convenience exports which are needed by some + Adeos domains loaded as kernel modules. */ +EXPORT_SYMBOL(do_exit); diff -uNrp linux-2.6.13/kernel/exit.c linux-2.6.13-adeos/kernel/exit.c --- linux-2.6.13/kernel/exit.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/exit.c 2005-09-04 12:38:02.000000000 +0200 @@ -833,6 +833,9 @@ fastcall NORET_TYPE void do_exit(long co del_timer_sync(&tsk->signal->real_timer); acct_process(code); } +#ifdef CONFIG_ADEOS_CORE + __adeos_exit_process(tsk); +#endif /* CONFIG_ADEOS_CORE */ exit_mm(tsk); exit_sem(tsk); diff -uNrp linux-2.6.13/kernel/fork.c linux-2.6.13-adeos/kernel/fork.c --- linux-2.6.13/kernel/fork.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/fork.c 2005-09-04 12:38:02.000000000 +0200 @@ -1115,6 +1115,14 @@ static task_t *copy_process(unsigned lon nr_threads++; total_forks++; write_unlock_irq(&tasklist_lock); +#ifdef CONFIG_ADEOS_CORE + { + int k; + + for (k = 0; k < ADEOS_ROOT_NPTDKEYS; k++) + p->ptd[k] = NULL; + } +#endif /* CONFIG_ADEOS_CORE */ retval = 0; fork_out: diff -uNrp linux-2.6.13/kernel/irq/handle.c linux-2.6.13-adeos/kernel/irq/handle.c --- linux-2.6.13/kernel/irq/handle.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/irq/handle.c 2005-09-04 12:38:02.000000000 +0200 @@ -117,13 +117,30 @@ fastcall unsigned int __do_IRQ(unsigned /* * No locking required for CPU-local interrupts: */ +#ifdef CONFIG_ADEOS_CORE + if (!adp_pipelined) + desc->handler->ack(irq); + + /* If processing a timer tick, pass the original regs + as collected during preemption and not our phony - + always kernel-originated - frame, so that we don't + wreck the profiling code. */ + + if (adp_pipelined && __adeos_tick_irq == irq) + action_ret = handle_IRQ_event(irq,__adeos_tick_regs + smp_processor_id(),desc->action); + else +#else /* !CONFIG_ADEOS_CORE */ desc->handler->ack(irq); +#endif /* CONFIG_ADEOS_CORE */ action_ret = handle_IRQ_event(irq, regs, desc->action); desc->handler->end(irq); return 1; } spin_lock(&desc->lock); +#ifdef CONFIG_ADEOS_CORE + if (!adp_pipelined) +#endif /* CONFIG_ADEOS_CORE */ desc->handler->ack(irq); /* * REPLAY is when Linux resends an IRQ that was dropped earlier @@ -168,6 +185,11 @@ fastcall unsigned int __do_IRQ(unsigned spin_unlock(&desc->lock); +#ifdef CONFIG_ADEOS_CORE + if (adp_pipelined && __adeos_tick_irq == irq) + action_ret = handle_IRQ_event(irq,__adeos_tick_regs + smp_processor_id(),action); + else +#endif /* CONFIG_ADEOS_CORE */ action_ret = handle_IRQ_event(irq, regs, action); spin_lock(&desc->lock); diff -uNrp linux-2.6.13/kernel/panic.c linux-2.6.13-adeos/kernel/panic.c --- linux-2.6.13/kernel/panic.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/panic.c 2005-09-04 12:38:02.000000000 +0200 @@ -77,6 +77,9 @@ NORET_TYPE void panic(const char * fmt, va_end(args); printk(KERN_EMERG "Kernel panic - not syncing: %s\n",buf); bust_spinlocks(0); +#ifdef CONFIG_ADEOS_CORE + __adeos_dump_state(); +#endif /* CONFIG_ADEOS_CORE */ /* * If we have crashed and we have a crash kernel loaded let it handle diff -uNrp linux-2.6.13/kernel/printk.c linux-2.6.13-adeos/kernel/printk.c --- linux-2.6.13/kernel/printk.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/printk.c 2005-09-04 12:38:02.000000000 +0200 @@ -502,6 +502,66 @@ __setup("time", printk_time_setup); * is inspected when the actual printing occurs. */ +#ifdef CONFIG_ADEOS_CORE + +static raw_spinlock_t __adeos_printk_lock = RAW_SPIN_LOCK_UNLOCKED; + +static int __adeos_printk_fill; + +static char __adeos_printk_buf[__LOG_BUF_LEN]; + +void __adeos_flush_printk (unsigned virq) +{ + char *p = __adeos_printk_buf; + int out = 0, len; + + clear_bit(ADEOS_PPRINTK_FLAG,&adp_root->flags); + + while (out < __adeos_printk_fill) { + len = strlen(p) + 1; + printk("%s",p); + p += len; + out += len; + } + __adeos_printk_fill = 0; +} + +asmlinkage int printk(const char *fmt, ...) +{ + unsigned long flags; + int r, fbytes; + va_list args; + + va_start(args, fmt); + + if (adp_current == adp_root || + test_bit(ADEOS_SPRINTK_FLAG,&adp_current->flags) || + oops_in_progress) { + r = vprintk(fmt, args); + goto out; + } + + spin_lock_irqsave_hw(&__adeos_printk_lock,flags); + + fbytes = __LOG_BUF_LEN - __adeos_printk_fill; + + if (fbytes > 1) { + r = vscnprintf(__adeos_printk_buf + __adeos_printk_fill, + fbytes, fmt, args) + 1; /* account for the null byte */ + __adeos_printk_fill += r; + } else + r = 0; + + spin_unlock_irqrestore_hw(&__adeos_printk_lock,flags); + + if (!test_and_set_bit(ADEOS_PPRINTK_FLAG,&adp_root->flags)) + adeos_trigger_irq(__adeos_printk_virq); +out: + va_end(args); + + return r; +} +#else /* !CONFIG_ADEOS_CORE */ asmlinkage int printk(const char *fmt, ...) { va_list args; @@ -513,6 +573,7 @@ asmlinkage int printk(const char *fmt, . return r; } +#endif /* CONFIG_ADEOS_CORE */ asmlinkage int vprintk(const char *fmt, va_list args) { diff -uNrp linux-2.6.13/kernel/sched.c linux-2.6.13-adeos/kernel/sched.c --- linux-2.6.13/kernel/sched.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/sched.c 2005-09-10 17:03:24.000000000 +0200 @@ -1474,6 +1474,13 @@ static inline void prepare_task_switch(r { prepare_lock_switch(rq, next); prepare_arch_switch(next); +#ifdef CONFIG_ADEOS_CORE + { + struct { struct task_struct *prev, *next; } arg = { current, next }; + __adeos_schedule_head(&arg); + adeos_hw_cli(); + } +#endif /* CONFIG_ADEOS_CORE */ } /** @@ -1533,6 +1540,9 @@ asmlinkage void schedule_tail(task_t *pr #endif if (current->set_child_tid) put_user(current->pid, current->set_child_tid); +#ifdef CONFIG_ADEOS_CORE + __adeos_enter_process(); +#endif /* CONFIG_ADEOS_CORE */ } /* @@ -2793,6 +2803,10 @@ need_resched_nonpreemptible: } else run_time = NS_MAX_SLEEP_AVG; +#ifdef CONFIG_ADEOS_CORE + if (adp_current != adp_root) /* Let's be helpful and conservative. */ + return; +#endif /* CONFIG_ADEOS_CORE */ /* * Tasks charged proportionately less run_time at high sleep_avg to * delay them losing their interactive status @@ -2907,6 +2921,21 @@ switch_tasks: prepare_task_switch(rq, next); prev = context_switch(rq, prev, next); barrier(); +#ifdef CONFIG_ADEOS_CORE + if (adp_pipelined) + { + __clear_bit(IPIPE_SYNC_FLAG,&adp_root->cpudata[task_cpu(prev)].status); + adeos_hw_sti(); + } + + if (__adeos_schedule_tail(prev) > 0 || adp_current != adp_root) + /* Someone has just recycled the register set of + prev for running over a non-root domain, or + some event handler in the pipeline asked for a + truncated scheduling tail. Don't perform the + Linux housekeeping chores, at least not now. */ + return; +#endif /* CONFIG_ADEOS_CORE */ /* * this_rq must be evaluated again because prev may have moved * CPUs since it called schedule(), thus the 'rq' on its stack @@ -3558,6 +3587,21 @@ recheck: task_rq_unlock(rq, &flags); goto recheck; } +#ifdef CONFIG_ADEOS_CORE + { + struct { + struct task_struct *task; + int policy; struct sched_param *param; + } evdata = { + p, policy, param + }; + if (__adeos_renice_process(&evdata)) + { + task_rq_unlock(rq, &flags); + return 0; + } + } +#endif /* CONFIG_ADEOS_CORE */ array = p->array; if (array) deactivate_task(p, rq); @@ -5263,3 +5307,60 @@ void normalize_rt_tasks(void) } #endif /* CONFIG_MAGIC_SYSRQ */ + +#ifdef CONFIG_ADEOS_CORE + +int __adeos_setscheduler_root (struct task_struct *p, int policy, int prio) +{ + prio_array_t *array; + unsigned long flags; + runqueue_t *rq; + int oldprio; + + if (prio < 1 || prio > MAX_RT_PRIO-1) + return -EINVAL; + + rq = task_rq_lock(p, &flags); + array = p->array; + if (array) + deactivate_task(p, rq); + oldprio = p->prio; + __setscheduler(p, policy, prio); + if (array) { + __activate_task(p, rq); + if (task_running(rq, p)) { + if (p->prio > oldprio) + resched_task(rq->curr); + } else if (TASK_PREEMPTS_CURR(p, rq)) + resched_task(rq->curr); + } + task_rq_unlock(rq, &flags); + + return 0; +} + +EXPORT_SYMBOL(__adeos_setscheduler_root); + +void __adeos_reenter_root (struct task_struct *prev, + int policy, + int prio) +{ + finish_task_switch(this_rq(),prev); + if (reacquire_kernel_lock(current) < 0) + ; + preempt_enable_no_resched(); + + if (current->policy != policy || current->rt_priority != prio) + __adeos_setscheduler_root(current,policy,prio); +} + +EXPORT_SYMBOL(__adeos_reenter_root); + +void __adeos_schedule_back_root (struct task_struct *prev) +{ + __adeos_reenter_root(prev,current->policy,current->rt_priority); +} + +EXPORT_SYMBOL(__adeos_schedule_back_root); + +#endif /* CONFIG_ADEOS_CORE */ diff -uNrp linux-2.6.13/kernel/signal.c linux-2.6.13-adeos/kernel/signal.c --- linux-2.6.13/kernel/signal.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/signal.c 2005-09-04 12:38:02.000000000 +0200 @@ -613,6 +613,13 @@ void signal_wake_up(struct task_struct * set_tsk_thread_flag(t, TIF_SIGPENDING); +#ifdef CONFIG_ADEOS_CORE + { + struct { struct task_struct *t; } evdata = { t }; + __adeos_kick_process(&evdata); + } +#endif /* CONFIG_ADEOS_CORE */ + /* * For SIGKILL, we want to wake it up in the stopped/traced case. * We don't check t->state here because there is a race with it @@ -882,6 +889,17 @@ specific_send_sig_info(int sig, struct s BUG(); assert_spin_locked(&t->sighand->siglock); +#ifdef CONFIG_ADEOS_CORE + /* If some domain handler in the pipeline doesn't ask for + propagation, return success pretending that 'sig' was + delivered. */ + { + struct { struct task_struct *task; int sig; } evdata = { t, sig }; + if (__adeos_signal_process(&evdata)) + goto out; + } +#endif /* CONFIG_ADEOS_CORE */ + if (((unsigned long)info > 2) && (info->si_code == SI_TIMER)) /* * Set up a return to indicate that we dropped the signal. diff -uNrp linux-2.6.13/kernel/sysctl.c linux-2.6.13-adeos/kernel/sysctl.c --- linux-2.6.13/kernel/sysctl.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/kernel/sysctl.c 2005-09-04 12:38:02.000000000 +0200 @@ -999,6 +999,9 @@ void __init sysctl_init(void) #ifdef CONFIG_PROC_FS register_proc_table(root_table, proc_sys_root); init_irq_proc(); +#ifdef CONFIG_ADEOS_CORE + __adeos_init_proc(); +#endif /* CONFIG_ADEOS_CORE */ #endif } diff -uNrp linux-2.6.13/lib/smp_processor_id.c linux-2.6.13-adeos/lib/smp_processor_id.c --- linux-2.6.13/lib/smp_processor_id.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/lib/smp_processor_id.c 2005-09-04 12:38:02.000000000 +0200 @@ -8,9 +8,21 @@ unsigned int debug_smp_processor_id(void) { +#ifdef CONFIG_ADEOS_CORE + unsigned long preempt_count; + int this_cpu; + cpumask_t this_mask; + + if (adp_current != adp_root) + return adeos_processor_id(); + + preempt_count = preempt_count(); + this_cpu = raw_smp_processor_id(); +#else /* CONFIG_ADEOS_CORE */ unsigned long preempt_count = preempt_count(); int this_cpu = raw_smp_processor_id(); cpumask_t this_mask; +#endif /* CONFIG_ADEOS_CORE */ if (likely(preempt_count)) goto out; diff -uNrp linux-2.6.13/mm/vmalloc.c linux-2.6.13-adeos/mm/vmalloc.c --- linux-2.6.13/mm/vmalloc.c 2005-08-29 01:41:01.000000000 +0200 +++ linux-2.6.13-adeos/mm/vmalloc.c 2005-09-04 12:38:02.000000000 +0200 @@ -18,6 +18,9 @@ #include #include +#ifdef CONFIG_ADEOS_CORE +#include +#endif /* CONFIG_ADEOS_CORE */ DEFINE_RWLOCK(vmlist_lock); @@ -148,10 +151,17 @@ int map_vm_area(struct vm_struct *area, pgd = pgd_offset_k(addr); spin_lock(&init_mm.page_table_lock); do { +#ifdef CONFIG_ADEOS_CORE + pgd_t oldpgd = *pgd; +#endif /* CONFIG_ADEOS_CORE */ next = pgd_addr_end(addr, end); err = vmap_pud_range(pgd, addr, next, prot, pages); if (err) break; +#ifdef CONFIG_ADEOS_CORE + if (pgd_val(oldpgd) != pgd_val(*pgd)) + set_pgdir(addr, *pgd); +#endif /* CONFIG_ADEOS_CORE */ } while (pgd++, addr = next, addr != end); spin_unlock(&init_mm.page_table_lock); flush_cache_vmap((unsigned long) area->addr, end);